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I   IlffiRODUCTION* 


It  is  frequently  necessary  for  many  problems  In  applied  mathematics 
to  solve  large  systems  of  linear  algebraic  equations.  These  equations  arise 
Often  from  statistical  data  and  finite  difference  approximations  to  continuous 
problems.  For  problems  of  the  latter  kind  the  matrix  of  coefficients  contain 
very  few  non-zero  elements.   It  is  not  unkno™  for  the  number  of  equations  to 
be  as  large  as  10,000.   For  such  problems  iterative  methods  are  often  used  for 
solving  the  system  of  equations. 

In  recent  years,  a  number  of  iterative  methods  have  been  proposed'  for 
solving  the  system  of  equations 


Ax  =  y 


where  y  is  a  known  N  ^  order  vector,  A  is  a  known  matrix  of  order  N,  and  ^  is 
the  unknown  N^^  order  vector.  For  a  particular  finite  difference  approxi:nation 
for  partial  differential  equations  of  the  elliptical  t^e,  it  has  been  shown  by 
Young  [93  that  the  matrix  of  coefficients  under  a  suitable  permutation  appears 
thusly: 


h 


(1.1) 


Where  D^  and  D^  are  N^xN^  and  N^x  N^  diagonal  matrices,  respectively.   If  there 
exists  a  permutation  matrix  P  such  that 


P  A  P  =  A' 


And  A-  has  the  form  of  (l.l)  then  A  is  said  to  have  property  (A).   If  a  matrix 
with  property  (A)  has  the  form  of  (l.l)  then  it  is  said  to  have  property  (A) 
*  III   ^peS^l'"'  "°''''°"  "^"^  '"  ^^^^  ^^^  subsequent  chapters  is  defined  in 
Numbers  in  brackets  refer  to  the  references  cited  in  the  bibliography. 


with  ordering  o-^.     The  purpose  of  this  investigation  is  to  show  the  relation, 
between  two  iterative  methods,  the  modified  Chebyshev  method  (see  below)  and 
the  successive  overrelaxation  method,  when  the  raatrix  A  is  symmetric,  positive 
definite  and  possesses  property  (A)  with  ordering  cT  . 

In  chapter  II  we  give  the  derivation  of  von  Neumann  [3]  of  the 
Chebyshev  iteration  scheme  which  is  as  follows: 

/k.l  =  ^\,l   (f,  -   Sy"-^k.l)  -^1   (k>  1)  (l.2.a) 

'l     -     io-^y  (l.2.b) 


b  1 


k+1  ~  ^   -2 
2  -  \^b, 
k 


h  =  1 


I 

where  \  is  the  largest  proper  value  of  G.   G  and  H  satisfy  the  relation 


G  +  H  A  =  I. 
It  is  shown  that  if  H  is  positive  definite,  then 

1 

cosh  a;  =  ■-—  . 

X 

In  addition,  we  consider ( 1, 2  .a  )when  h^^^   =  b  for  all  k;  this  itera- 
tion scheme  is  called  the  one  parameter  iteration  process.   It  is  shown  that 

(    j  =  1, ...,N 
.|2b-i|  0^-1)  ^Ma.^^    |^,/-2  1 13-^^1 1  j         (1.3) 

j  =  1, ..,n  ^ 

where  0^^  (i  =  1^2;  j   =  1, ...,n)  are  the  roots  of  the  polynomial 


0^  -  2bA..0.  +  (2b  -  1)  =  0 


and  X.    (J  =  1, ...,  N)  are  the  proper  values  of ,G.   For  0  <  b  <  1,  |0. .|  <  1, 
and  hence  | |x  -  ^^| |  ^  0  as  k  -^  -   By  a  theorem  of  Frankel[l], 


Max        |0i,(b)| 
1=1,2  ^J 


j=l,  ...,N 
Is  minimized  with  respect  to  b  when 


1  +  Jl-X'^ 
and  it  is  also  shown  that  b^  (k  >  l)  decreases  with  k  monotonically  to  ^. 

The  bound  given  by  (l. 3)  is  actually  attainable  and  consequently  for 


b  =  b,     max 


.->  ^1  t  _.  I  1^  ~  ^kl  I  °^^y  increase  for  small  k.  When  ^  =  g/  +  Hy, 


ii5-;ji=ai 


however,  for  the  one  parameter  iteration  scheme 

f  II    /|h|  y/2 


where 

/^  =  /  2S  -  1  =       ^, 

^  1.  vC^ 

and,  hence,     max      I  1^  "  5,1  I  ^^^*  decrease  as  k  increases. 

In  chapter  III  we  use  a  device  of  Riley  [4] to  modify  the  Chebyshev  it- 
eration scheme  and  the  one  parameter  iteration  process.   If  G  =  j  -  d"^  A  where 
D  is  a  diagonal  matrix  whose  non-zero  elements  are  the  diagonal  elements  of  A 
then  for  matrices  which  are  symmetric,  positive  definite  possess  property  A  and 
ordering  o-^,    it  is  possible  to  modify  the  Chebyshev  iteration  scheme  and  one 
parameter  iteration  scheme  so  that  after  k  iterations,  we  have  computed 


;. 


^2k  has  N^  components  and  ^^^^^  ^^^   ^^  components  (n^  +  n^  =  N), 

For  the  modified  iteration  scheme, 
.-1  /^ 


J.=  -.=-^(?-^.-U^.,-B^.,).)^^., 


(l.'i-  ■) 


where 


L  = 


0 


D  = 


r       _ 


^2:-" 


C     T 


and  If  ^  -  G^Q  +  Hy;^  then 

If  c^  =  b^  with  b^  defined  as  for  (l.2.c)  then  (l.k)   descr-^bes  the  modified 
Chebyshev  method  and  if  c^  .  %,   then  (1.4.0  describes  the  successive  over  .. 
relaxation  method.   Whereas  the  Chebyshev  iteration  scheme  and  the  one  para- 
meter iteration  scheme  require  the  two  previous  approximants  to  compute  the 
new  approximant,  the  modified  Chebyshev  method  and  successive  overrr-laxation 
method  only  reqirire  the  previous  appro-imant. 

For  matrices  which  have  property  (a)  and  ordering  c-   V^e   v-ov-r 
vectors  and  proper  values  satisfy  certain  simple  relationships.   U.injt  th-  S' 

th 


relationships,  we  are  able  to  show  that  if  j?-  is  the  k*^  iterate  of  the 

1/2 


modified  Chebyshev  method 

ll^-f^ll        /ID 


|5^^'-r'll 


:(i)  _  d(i)'i ,  ^  (  iDl 


I'u 


/ 


vhere   cosh  ^  =  ^  :-nd  ^   is  the  largest  proper  value  of  G  =  I  -  d"^  A.   If 
y^   is  the  k   iterate  of  the  successive  overrelaxation  method  when 


then 


f  ^  =  B-  (.-(^)  -  S/,X)) 


I  |3  -h\  I       /  |D  I  W2  y 

and 

/^=  /^l   -  tf 

<  1  +  /^  • 


Then  using  a  simple  argument,  we  are  able  to  show  that  for  k  fixed 

The  results  of  chapters  II  and  III  are  ^11  arrived  at  under  the 
assumptions  that  the  numerical  operations  have  been  performed  exactly.  When 
the  calculations  are  performed  on  a  high  speed  digital  computer  In  fixed  point 
arithmetic,  the  operations  of  multiplication  and  division  are  replaced  by 
pseudonnultlpllcatlon  and  pseudo  division.   In  chapter  IV  we  consider  the 
effect  of  the  roundoff  errors. 

For  the  Chebyshev  iteration  scheme  one  computes  the  sequence  [^  ). 
Under  some  simple  assumptions  about  the  numerical  operations,  it  is  shov^^ 

that 

■^  >  b^  >....>  bj; 

and  that  for  k  >  M-, 


\  =   Vi=  •  •  •  •    =  ^'• 

Thus  the.Chehyshev  iteration  scheme  degenerates  into  the  one  parameter  itera- 
tion scheme  and  similarly  the  modified  Chebyshev  method  degenerates  into  the 
successive  overrelaxation  method. 

We  assume  that  the  digital  numbers  are  represented  as  a  sign  vith 


^  digits  in  a  number  system  of  base_  p.  Then  within  the  computer  for  the 
Chebyshev  iteration  scheme  and  one  paraiaeter  iteration  scheme,  one  has 

/^k  "^  (k  =  0,  1,  ...) 

where  f  3    indicates  a  right  shift  of  p^  places  and 3^  is  a  scaling  para- 
meter chosen  so  that  the  elements  of  the  scaled  approximants  are  less  than 
one  in  magnitude.   It  is  supposed  that  f or  k  >  K 

^K  =  Vl  ^  •  "  •  =  P   • 
A  bomxd  is  given  for  mln  |  \1   -  (p.^   ^  ^P)  .  pP,  |  ,^^^^   increases  with  X' 
vhdl-e  X-  is  the  largest  proper  value  of  G,  the  digital  representation  of  G. 

Likewise  for  the  modified  Chebyshev  method  and  successive  over- 
relaxation  method,  one  computes 

^  .  '  ^k 

Jk7^  (k  =  0,  1,  ...)  . 

A  bound  is  given  for  min  Mx  -  (7  -  rP^  ftP|  I  ttv,-;  v,  •  - 

B      ^x  laxii  I  I A   Y^   ,  p  ;  f3  I  I  which  increases  as  jj.' 

approaches  one  where  m'   is  the  largest  proper  value  of  G,  the  digital 
iNepre^entation  of  G  =  I  -  d"''"A. 

In  chapter  V,  a  numerical  example  is  given  in  which  the  modified 
Chebyshev  method  and  two  variants  of  the  successive  overrelaxation  method 
are  used.   For  variant  on.,  f^^^   .   d'^J^^)  _  3^(1)^.  ^^^  ^^^.^^^  ^^^^ 

there  is  no  special  relationship  between  J^^^  and  f^^\      it  is  seen 
that  the  analytical  results  of  chapter  III  are  vitiated  by  roundoff  error. 
Nevertheless  we  conclude  that  the  modified  Chebyshev  method  should  be  used 
to  solve  linear  equations  for  which  the  matrix  A  is  symmetric  positive 
definite  and  possesses  property  (A)  with  ordering  a-. 


II   ON  AVERAGING  PROCESSES 

2.1  Linear  Iterative  processes 

We  consider  a  system  of  linear  algebraic  equations  in  N  unknowns 

A5  =  ^  (2.1) 

where  A  is  a  known  square  matrix  of  rank  and  order  N;  and  if  and  f  are  vectors 
of  order  N.   If  A  is  not  symmetric,  then  we  can  multiply  (2.1)  on  the  left  so 
that  A  Ax  =  A  3^,  and  now  A  A  is  symmetric  and  positive  definite.   Hence^  we 
assume  here  and  in  the  following  that  A  i£  symmetric  and  positive  definite . 

As  has  been  pointed  out  by  von  Ne\miann  in  [  31,  a  large  number  of 
iteration  schemes  for  solving  (2.1)  are  described  by  the  equations 

If  ?     =   X,  then  we  requiic  f.-,    =  x,  and,  hence, 


t  =  G^5^  +  \y 


=  G,  X  +  H,  Ax  . 
k     k 


(2.3) 


A  necessary  and  sufficient  condition  that  (2.3)  hold  true  for  all  5f  is  the- L 

k    iSL 
We  shall  assume  that  this  conditioii  Lolds  true  in  the   following.   Furthermore, 

-'  fk = fk+i'  ^'^^'^  h  -  °kfk  *  V  ^>- 


8 


However,  from  (2,k-)      (l   j.  a   )   -  tj  &   .,   ^     ^  ,        x  ^ 

^  ^    ^  '^^   U   Gj^;  -  Hj^A  and,  hence,  (2.5)  becomes  H^^A^  = 

^^^""fk  =  /k+1  implies  /k  =  ^  i^ 


V- 


^^^  K\^^      .  (2.6) 


We: shall  assume  that  this  condition  is  satisfied. 
Nov  subtracting  (2.2)  from  (2.3)  ve  have 


The  solution  of  this  matrix  difference  equation  i 


If  Gj^  -  G  for  all  k,  then  (2.7)  becomes 

in  ^rder  for  the  error  vector,  1  -J^,   to  tend  to  .ero  1„  (2.8),  the  initial, 
errar  vector,  {t  -  ^ ^) ,   „,,t  te  In  the  .anifold  generated  by  the  proper  and 
invariant  vectors  of  G  associated  with  the  proper  values  which  are  less  than 
one  ln^„agnltude.  However,  since  we  do  not  have  any  a  priori  information 
about  3  -f^,    in  general,  we  shall  assume  that  all  the  proper  values  of  G  are 

less  than  one  In  magnitude,  and,  hence  t  -  9     wi  1 1  +»„/.  * 

,  ..V.I,  iiei.ce,  ^  -  y^   will  tend  toward  zero  inde- 
pendently of  1   -  ^^.  Then  since  H  =  (I  -  GIa'^  by  (£.3), 

K  >  rank  H  >  mln[rank  (l  -  G),  rank  A"^]  =  H 

and,  thus,  rani  E  -  S  and  H  ^  exists.  Note  that  for  u  =  ,.,  (k  >  o),  the  con- 
dition that  an  the  proper  values  of  G  be  less  than  one  ln'"magnltude  implies 
(£.6). 


2,2  The  averaging  process 

Let  us  now  ccr.sider  the  ixeratire  process  ^    -  g7   +  ^^      ^ 
f^   OS  /k+1  -  ^/k  +  Hy-   From 

(2.aj,  we  see  that  if  the  largest  pr-op^r  va^r*-  of  r  •>, 

e    P-oper  va^ue  of  C-  in  magnitude  is  near  one, 

the  iterative  process  may  be  slow  to  coiig<=rge   Tr  f  ^1 

t-Liigergfe..   in  [  3J,  von  Neumann  <3onsider3 

replacing  the  original  sequence  /   ?      >.v  .  * 

^  So'S'l'    •••  ^y  ^  Sequence  of  averages 

/'O^A'  •••  where 
By  (2,8)  then  .        ^   ^ 


Tha^  is,  afts''"  k  it>--^a  +  Tn->c  -r-^  t-v^ 

^   J.  o>-^  A  LO'^_aT:ions  m  the  new  T)roce'-iq  ^'^^.i  ^•r^•5^■-,•^^ 

i^jLceob,  ^_.e  initial  error  is  multi- 
plied by  a  matrix  polynomial  so  that  ' 

■'■-  V  =  ^'  *''^"  "e  i^-'e  t!is  original  process  and  P,(g)  =  a''. 

Then  for  a  given  matrix  G,  ve  wish  t-^  dot  — i-i^  +^r. 

,      c  WXK..I  ov.  ae-csij^lne  the  necessary  and 

s-,;.fficient  conditions  necessary  fo-  11m  P  fal  -  n   mv 

"  tTo/^         ~  answer  is  well  knowji 

and  stated  in  [3].   Let  X.  (i  ..  i  2  ^"^  m<  -^  i.«  ^v 

i  V  -   ±,  ^,  . . .  ,m  <  nj  be  .he  proper  values  of  G,  and 

e.   (i  =  1,  2,  ...)  be  the  order  of  the  elementary  divisor  of  G  that 
corresponds  to  X.,      Let  the  r^^  derivative  of  P^^(z)  be  denoted  as  p(-)(.). 
Then  the  ncefesary  and  sufficient  condition  that  lim  P  (g)  =  0  is  that 

■I         j\ 


k^-  ^ 


10 

m  P  ^  (x^)  .  0  for  all  those  coablnatloas  1(=  i,  2,  ...  „),  ,(,  ^^  ,^  __  ^ 

for  which  e.  >  r. 

Then  the  choice  of  polynomial  P^(g)  „iii  depend  on  the  distribution 

Of  the  proper  values  and  the  order  of  the  el«neata.y  divisors.  It  has  been 

sho™  by  Varga  t6]  that  if  the  only  tao.ledge  about  the  roots  Is  that  they 

lie  in  a  circle  around  the  origin  in  the  complex  plane  (i.e.  \K^\  <  ^ 

1  =  1.,  2,    ...  n),  the  best  choice  of  P  (g)  Is  P  (r)       r^   tt   ,.  . . 

^j^V<^;  IS  ■f^l.Cr;  =  G  .  We  shall  assume 

that  the  proper  values  of  G  are  real  and  satisfy  |\  |  <  x. 

Let  us  assume  now  that  H  is  real,  symmetric,  and  positive  definite 
Since  this  covers  certain  important  applications  and  allows  the  use  of  some 
rather  effective  methods.  Therefore,  there  exists  a  matrix  F  such  that 
f/  =^H.  Since  H  and  A  are  symmetric,  E'^   -  A  is  symmetric  a^d  likewise 
F  (H-1  -  A)F.  Then  there  exists  an  orthogonal  transforation  0  such  that 

fV^-a)f  =  oao   ,  (,^^^) 

Where  A  is  the  diagonal  matrix  of  proper  values  of  F^H'^  -  A)F.   Pre- 
multiplying  (2.10)  by  F  and  postmultiplying  by  F'^,  we  have 

FF*(H"^  -  A)  =  FOAO*F"^ 
so  that 

H(H"^  -  A)  =  F07lO*F"^  . 
But  S{r^   -  A)  =  I  -  HA  =  G  by  (2.4)  and  thus 

G  =  FQA0*F"-^  .  , 

(2.11) 

Then  (2.11)  expresses  G  in  the  Jordan  canonical  form  with  A  the  matrix  of 
proper  values  of  G.  Since  vl  is  diagonal,  the  elementary  divisors  of  G  are 
simple  and  real. 


Substituting  (2.11)  into  (2.9),  we  see  that 


X 


Since  (2.11)  yields  G  in  the  Jordan 


canonical  form.  G  =  FO-A''b*F~''". 


Thus 


P.(G)  =  Z 


V  FO^'^0*F"^  =  F0|  E  a  aWo*F"^ 


=  FOR  (7l)0*F~-'- 
k 


Because  TV  is  a  diagonal  matrix. 


P,(-A) 


p^(\^)   0 


^    ^Ic^^n) 


Now  (5r  -|^)  .  ?^(G)  (5e  -  ^q)  .  FOP^(yl)0*F-l(?  -^^)  and  hence  by  (A.2), 
(A. 6),  (A.8),  (A.9),  (A, 10),  (A. 11) 


ii^-r.iiii 


H 


1/2 


u 


ihT/ 


Max 
i=l,2. 


,N 


I  P.O.,) 


X  - 


/o 


(2.12) 


Naturally,  we  wish  to  choose  P^(x)  so  that   Max     |P  (?^.)|-^0  as  k -^  ^  . 

i=l  2    . ,N 
Since  the  elementary  divisors  of  G  are  simple/ the 'choice  of  P  (\)  will  depend 

on  the  distribution  of  the  \. 's.   ¥e  shall  assume  that  we  do  not  have  any 

exact  knowledge  of  ,\^,  ...  K^   i,ut  simply  the  information 
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-X  <  X.   <  X 

—     1  — 


(1  =  1,  2,     ...  i^) 


(2.13) 


If  a  <  \^  <  \^  . , ,  <  \^  <  b,  then  by  a  suitable  linear  transformation,  (2.I3) 

will  hold  true.   Then  for   | |?  - J^l |   fixed,  we  wish  to  choose  P^(x)  so 

that  MaxJPj^(\)|  is  minimized  subject  to  the  condition  P,  (l)  =  l  since 
|/c  |<X  k 


k 


=0 


It  is  veil  known,  ef.  [3],  that  the  desired  polynomial  Is 


(2,14-) 


where 


1  r._  .  ,r2 


\i-)   =     i[(z  Wz2  .1   )^^  (,  ./;2-77  ^k  J 


th 


thek   Chebyshev  polynomial.   If  | z|  <  1,  thenl^(z)  =  cos  k  (arc  cos  z), 
and  for  |z|  >  i,  r^(z)  =  eosh  z(arc  cosh  z).  Thus 

cos  k  0  ;^ 

V^i^  ^  cosh  k  <^  ""^^^^  cos  Q.-  =  ~  and  cosh  6<^  =  -^ 


Then  (2.12)  becomes 

\l/2 


"^-^^"^(Sjj 


l-"-fol 


cosh  k  tu 

2.3  The  Chebyshev  Iterative  process 
^    k 
Since  p^  =^Z^  a^^^,  it  would  seem  that  it  is  necessary  to  store 

all  the  vectors  ^Q,  ' '  -  /k  i«  order  to  compute  the  vector  f^.     However, 
the  Chebyshev  polynomials  satisfy  simple  recursive  relationships  which  we 
now  proceed  to  take  advantage  of.  The  equations  developed  here  were 
originally  obtained  in  [3]. 
Since 
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\-^l^^)   -^  Vl^2^  =  2zR^(z) 
and,  hence, 

•'m(-)  =  2^V-)  -  \.l(^)   .  (2.15) 

The  Initial  conditions  are  R^(z)   =  1  and  Rj^(z)  =  z. 

Mow  we  use  (s.lt)  to  pass  to  P^(x.).  Then  (2.15)  becomes 

and  dividing  through,  by  E^^^i-^)^   we  have 

1   \         /  X, 


Mr     3x  H,/-^  H^A 


R  ,  /^   R,  .   ^ 


V.  (tj  ^  »„(tj  ■.  (tJ  "  Mr?  ^j 

Using  (2.14),  the  above  relationship  becomes 
and  therefore, 

^k.i(\)  =  ^iVA(^i)  -  ^\VA.i(\)  (2.1,) 


where 


.. .  ""'(t 


■^   XR^/4-\    ■  (2-18) 


From  (2.15),  we  have 


Ik 


\(^]    ^..  (^ 


_2        "KXJ  T'- 1 1      ^   ; 


9Q  tliat  using  (2.17) 

k+1  .  \"k+l       -^  (2.19) 


or 


1 


'k+1  2   _  ^2^      ^1  =  1      •  (2.20) 


Also  we  are  able  to  rewrite  (2.I7)  using  (2.I9)  so  that 
We  now  pass  back  to  the  original  process  which  yields  from  the  preceding 


equation 


\.X^^)   =  2\.l[°Pk(°)  -  P,-l(°)]  -  P,.i(G)   .  (2.21) 

Postmultlplytng  (2.21)  by  {1   -^^),  „e  have 
^k.l(°)  (?  -/o)  =  2\,i[GP,(G)  (?  -^g)  -  P^_^(G)  (Sr  .^;)] 

^Vl(0)(5-fo)  •  (£.22) 

Slr.ce  ?  -^  =  Pj^(G)  (J  -j*  ),  (2.22)  now  becomes 

^^  -^l'  =  '\*l''^^  -%^   -   (J  -  %.^l   ^(-^,.,)  .         (2.23) 
rToB  (S^>,  X-  -  Gx  =  Hy  so  that-  the  above  becomes 

fk.l  -  2\.l(sf,  ^  Hy*  -  ?,.,)  .  f^_^  (k  =  1,  2,  . .  .  )   (2.2U.a) 
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and  since 


=  X. 


''^(t^  ^ 


=^lled  the  Chebychev  Iteration  _scheme. 

2.4    MlE«££  the  errOT  after  k  Ueratlons 

For  the  Iteration  process  ?         =  o   ?    ^  u  ■* 

/k+1       \Jk  +  \y'   we  wish  to  determine 
an  upper  bound  for   1 1?  -  <?  I  I    *• 

,        .  -   11-      /,l  I    froM  the  numbers  which  have  been  computed  during 

the  Iteration  process.      Since 

we  have  by  (A.ll.b) 


In  addition,    since 


/k+i-/k  =  °k/k- V-fk 


16 
then  by  (A.ll.b) 


vould  slMply  need  to  Iterate  until  /kM  <  ^^  one 


We  note  that  I  I  -^  I  I  =  I  !^  1  j   n  •  ^ 

IKW    -  lt\f|  ve  W  When  ve  are  close  to  a  solution. 

On  the  other  hand,  for  the  averaging  process  It  ..   . 
+    ^,    M-^     -*  process  It  is  not  necessarily 

true  that   ^    ->||   n  --p   .  ^    _^ 

'  7k+l   ?'kl  I  =  0  If  and  only  if  I^  .  ^[  I  _  0   u 

^  '  Y k   ^fl  -0-  We  may,  however, 

compute  7     vex. 


and  thus  by  (A.ll.b) 


\t 


The  residual  vector  r     QH--?n  v   ^-i  .^ 

,  ,.  ,  ,  -^  ^''  '^"^  ^•'^  5"^^-*^'  i  l?,l  I  -  0  If  and  only  If 

I  \r^\  I  =  0.  Then  for  exact  calculations,  to  insure  1  I?  -  3  I  I  <  e 

„ .   -,  '  '         /]rl\   S^?   one  would 

simply  need  to  iterate  until  ^ 


mm 


<  £ 
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2.5  The  behavior  of  the  b  's 

^For  the  Chebyshev  it^rati6n  sch#te,  it  is  hecessary  to  compute  b^ 

for  each  ^    by  the  relationship  b  ,  ^  i-—    v    .   t,     ,.   ,  ,„, 

f  ^  ^     2   5C^  ""   1-  ^        equation  (^.18), 


k 


vf 


''^^'''^  '\(  -r-  )  -  cosh  k  ^,  cpsh  ^  =  -4c 

^  5: 


Since  cosh  oo  =  -■  '*'  ?, 1- 


6*^=  J- +    Ji'->? 


(2.25) 


Theorgn  (s.l):  The  b^'s  fo™  a  strictly  monotonlcally  decreasing  se^ence 
Whose  limit  is  '^  =  - 


1  +  /l  -  \^ 


Proof:        We  have  b,    >  b  if 

k         k+1  ^^ 


"•-''ii  .  -^(t 


XR,    /-J^l  XR,       /J^ 


or 

coshU  -  l)^.cosh(k  +  1)1^  -  cosh^  k  o;  >  0      .  ^  (2.26) 

But   cosh(k  -  l).t/cosh(k  +  1)^"=   cosh^k  C^  cosh^  u/  -  sinh^  k  o^^sl^h^o; 
ao  that   (2.26)  becomes 

CQsh^  k  c^  cosh^Lo  -   sinh^  k  uj  Blni?  u;  -   cosh.-  k  cO  >  0 
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or  since  cosh^  to   _  sinh^  i^  =  \ 


1    fjj(  n^c\n  U  //;     _•   •■  2 


sinh  ^(cosh  k  ^  -  ^inh^  k  uj)  =       sinh^  ^^  >  o 

1  +  e-2^ 


Since      b    =  e   +  p> ^__     e 

k+1   ^j^(k+l)a.;^-(k+i)u,J  =  -^ 


\ 


1  +  e' 


■2(k+l)^ 


andO<e-^<l,   l^^^l  =  ^-   However,  t^y.  (2.25) 


1  +  /l  -  }? 


and  thus  11m  b     =  L _  ^ 

k->-  ^-^1     i  +  /rr^       • 

2.6  A  one  parameter  iteration  process 

As  was  Shown  in  the  last  section,  the  b^'s  form  a  monotonically 
decreasing  sequence  and  converges  to  a  limit  ^.   Let  us  denote  the  machine 
generated  b^'s  as  b^.   Then  it  will  be  shown  in  Chapter  IV  that  under  suit- 
able assumptions  for  some  k>M,b=:b     -  ^.i 

-  •"  ""m    M+1  ~   =  °  •   Hence  the 

Chebychev  Iteration  process  when  carried  out  on  a  computer  ultimately  de- 
generates to  a  process  in  v^ich  b^  =  b.   In  this  section  we  discuss  such  a 
process  ab  initio. 

From  (2.11),  we  have  the  elementary  divisors  of  G  are  simple,  and 
thus  we  have 

G  =  m-^  (2.27) 

Where  JL   is  the  diagonal  matrix  of  proper  values.   We  wish  to  consider  the 
iteration  formula 
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Where  ^^  and  f^   are  given.  The  computation  scheme  described  by  (s.28)  will 
be  called  the  one  paj^meter  Iteration  scheme.  Then  from  (2.23),  we  have 

(^^-^.l)=2^[°(^-;,)-(2-^,.,)l.(?.f^.^)    (,>i)    (2.,5j 

and  thus  substituting  (2.27)  into  (2.29), 

Premultiplying  by  Q'^  and  setting  u^  =  Q"^(^  .  ^  ) 

\+l  =  2bA^  +  (1  -  2b)  \_^ 
or  since  A  is  a  diagonal  matrix 

•^J^k.l  =  2b"j,k  +  (1  -  St)  u.^^_^     (J  =  l,  2,  ...N)   (k>l)   . 

The  solution  of  this  difference  equation  Is  given  by  lemma  (A.l).  Returning 
to  the  vector  equation  (2.29),  we  have 

-   ^^  -  P   '   ^K«"'(?  -  %)    -   ^^^   ■   ^)  «S,-l«-'(?  -  %)  (2.30) 

IJ  "  hi  "        'IJ  '■  ''2J 


""'"'  [  'i^iio  =  jfr^.    "  ^  -  J  »^  ^1,  /  0, 


=  k?!^-'-       if  1  =  J    and  ^^_j  .  0g^ 
=0  If  i  /  J 

and  ^^j  and  0^^  are  the  roots  of  the  equation 

0f  -  2bXj0j  ^  (2b  -  1)  =  0    (J=l,  2,  ...H)   .       (2.31) 
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Thus   since  Q  =  FO  by  (2.11)   anri'  TTir*       it  ^  .       ,         .      , 

^J   K^.±±)  and  FF     =  H,    we  h^ve  by  (A.l),    (A, 2), 

(A,3),    (A.6),    (A.8),    (A,9),    (A.IO)   and  (A.ll) 


'""^^"^(w7)"f,.,^-,JK)n 


+    jSb   -  ll  Max  Ifs  i      I    !i;r  _  2 


,.,^..JK}..lil^-^Jlj.  (..3.) 

f         -^  ^1   i      "     ^On  k-1 

Since  j  S,  {  .  .   =  -i^ isL     _       v     ^m     ^k-m-l         ^        , 

Max   |(SJ..|  <  K    M.x(|0^j|>^-1,    |^^.|H-1).     ,^^^  ^^^^  ^^  ^^^^^^^^  ^^^^^^^^^ 
when  0^^  =  0g^.     Thus  (2.32)  becomes 

1/2 


e 


j=l,2,  ..N 


+  |2b  -  l|  (k  -  1)   Max     10.  1^-2  |  |^  -^  |  | 

j=l.'2,  ..n  ,  -   ^ 

(2.33) 

Since  I  1^  -  ^0 1  I  and  I  1^  -  ^J  I  are  fixed,  ve  wish  to  detemine  b  so  that 

Max     10  (b)| 
i=l,2      ij 
j=l,2,  .,N 

is  minimized  with  respect  to  b.  The  problem  has  been  considered  by  Frankel 
[1]  and  he  has  shown: 

aeorem  (2.2):  If  x  1.  the  largest  proper  value  of  G  and  -X  is  the  smallest, 

then  for  b  =  t  =  —1_  ,   the   Max     |0  (b)|   Is  minimized  with 

±  +    vi  -  A.  i=l,  2 

j=l,2,.. 

respect  to  b  and 
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Furthermore,  for  1  >  b  >^,   |0 .  .(b)  |  =  (2b  -  l)V2_ 

Let  0^  =  b?:  +  ^^^  -  2b  +  1    and  0  =  bX  -  /^ 


2 ,  u  /v   -  2b  +  1 

<>       .  2r2 

-  yb  +  1  =  0  and  (^^    =  (a    , 


then  for  b  =  b,  b  X  -  2b  .  1  =  o  and  ^^  =  0^.  i^n  (2.33)  beco.es 

\l/2 


X  -  -- 


r  ^      ,        r-  (2.3t) 

where  p   =i'Sa  -  I.     a     i^. 


2,7  A  starting  procedure  for  the  one  parameter  iterat 


ion  process 


If  t   IS  the  proper  veetox  associated  with  -\,  and  t  -%     =  x  -  ^ 
^  ( o      /I 

=  ae,  then  from  (2.30) 

II"  -fi^ii  -(t%)    f^'"'^" " '"  ■  'V')  11^  -  foil 

we  see  that  although  ^'^\x  +  (k  -  l)/)l-^  0  a^  > -^  ^  +>,^ 

-o  ^   LX5.  -r  v-cl   x/^j-?  u  as  js -^  00,  the  maximimi  error 

may  actually  grow  during  the  early  iterations.   From  (2.30;,  we  note  that 

by  choosing  ^^  carefully,  it  may  be  possible  to  obtain  a  cancellation  of 

terms.   In  the  following  theorem,  we  give  a  means  of  accomplishing  this. 

Theory  (2.3):   If  ^^  =  G^^  +  ^,     and  if  1  >  b  >^, 

il^'-ill^lSjj'^^f-^^.jllx-foll 
wh&re  r>  =  i/2b  -  1  . 


Proof:  Since  ?  -  7,  =  G(i?  -  ^   .  ^,-l(^  .  y^    (,,3^)  ^^_^^ 

?-f,  =  Q[s^A.(£,.i)3^.^j«-V-p.  (2.35) 

Since  all  the  roots  of  (2.31)  are  equal 


m  modulus  for  1  >  b  >'S,  we  can 


represent  them  in  polar  form,  i.e. 


Plj  =V2b  -  1  e  J  =  (Oe  J 


02j  =\/2h  -  1  e   "^  =  ^e   ^ 

Hence  0^.  .  0^ .  =  2^  cos  9.  .  But  by  (2.31)  0^.  .  0^.  =  2bX.  so  that 

cos  9 . 
\.  = i 


Now  if  0^^  ^  0^^   (i  ^  1^  2,  ..n) 
^  ^  k-ljjj     r    sin  9.    b 


cos  9 .  -  c 


sin(k  -  1)9. 


sin  9. 
J 


L 


^1   sin  k9  cos  9.    sin(k  -  1)9 

-si  li_ j 


sin  9.    b 

J 


sin  9 


sln(k  -  1)9. 
But  since    i 


sin  k9 


sin  9 . 
J 


cos  9  -  cos  k9.  , 


sin  9^       j        J 


the  above  becomes 


-L    \  sin(k  -  1)9, 
t»    J        sin  9. 


~  +  -r—  cos  k9 . 


< 


'^-.)i...).^ 


since 


sin  k9. 
sin  9 . 


<  k 


Because  Sb  -  1  .  f ,   ^^   ,ee  the  above  becomes 


f 


.i^ 


1  +  — ^  k 
1  V 


If  the  roots  are  equal  In  magnitude,  then  the  above  Inequality  be- 
comes  an  equality.  Now  taking  bounds  of  (2.35),  we  have 


If  b  =  ^  = 


J. 
1  +  h  -1?   '  *^^^  ^^^  ^^°'^^  inequality  becomes 


'^-?^"^fSi7rV(-'^^"'-?°" 


By  the  following  calculation,  ve  show  that  i; 


=  G^  +  Hy  then 


Max 

X  " 


foll^o 


1"^   -^ 
X  -  - 


fkil 


\T^^ 


F 


miist  decrease  with  k  for  b  =  '^. 


Now 


|Hi,A^/2 


H 


'U 


u 


if 


2 


or 


\X{^-\)  +(^] 


1  -e; 

1  +  f  ^ 


<    1  - 


r 


But  since     0  <  /^  <  i^ 


2h 


since  f^,  =  G^g  +  ^, 


where 


and 


\^j_iG)   =  2bGPj^(G)  +  (1  _  2b)  PjG) 


Po(G)     =     I    and    F^(G)  =  G  . 
2.8  The  choice  of  h  when  f    =  Qp   +  jif 

Although  lb  minimizes  Max   10.  (b)  I  i^ith  rP^T.Pr.+  +^  >.-  •+  • 

i=l  2    ij    '      respect  to  b,  it  is  not  true 

J=1,2,.M 

that    Max      f !  x  -  ^  I  I   -?  o   •  •  •   , 

I  Ix-^  1  Uo   '   ^k"   ^^  mnDjmzed  with  respect  to  b  when 

7i  =  Gfo  -  =^  -nd  k  is  fixed.  For  J.  =  2,  the  Chebyshev  Iteration  sch^e 

yields  the  polynomial 

Sb^G^^  (1  .  2b2)l   . 
Thus  the  best  choice  of  b  when  two  itez^tions  are  perfox^ed  is  b  =  b^.  m 
ge4eral,  k  is  not  predete^ined  but  rather  one  iterates  until  either^]  |?J  | 
or  I |?J  i  is  sufficiently  small.  From  the  following  we  see  that  the  opti- 
mum b  is  arbitrarily  close  to  ^  for  k  large. 
Theorem  (2.4):  Let 


Pj^^^(\,b)  =  2bXP^(x,b)  +  (1  .  2b)  P^  rx,b) 


■with 

PqCX,!))  =  1       and       P^(\^b)  =  ^     . 
Then  for  ^^  =  G^Q  +  Hy, 

,   Max.     l^iK'^)] 
Proof?     From  theorem  (2.3)_, 

wjjare  ^  = /^  -  1 .  and 


Where  0   and  0   are  the  roots  of 


1.)     ^2j 


0^  -  2t)\  0  +  (2b  -  1)  =  0 


1.  Let  0  <  b  <  ^,  then 

0^  -0^ 

^^^^^  01  =  t)x  +  /t^^Tim 

02  =  t^  -  /b^'x^-  2b  +  1   . 
0-L  and  02  are  distinct  for  0  <  b  <'^  sinceb^  -  2^  +  1  =  0. 


Then  since 
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03_  +  $2^2  =  3bX, 


V^,b) 


0f  1 .  0^^-i 


'1       =^2 


1  /^r-^r 


2b 


2  ^r-^r 


?r^^--(i^A)'  1-^ 


Now 


Max_    IP  (\',^)|  ^ 

-     Max_    |P^(X,t)I   ^  TP^(X7b)T 

I  A,  |*^Ai 


21) 


^^1  +  k   /l    -  X^  ) 


k+1  ,        ,.    ,,   .k_i 


1  -  i^^/^^r'^    ^2 1 "  (y^i) 


1  -  ^  v^i)     "  ^2  rrry^ 


Now  0^  >  ^  Since  ^  minimizes  the  Max     |0,  ,  (b)  |,  and,  hence, 

1=1,2  ^ 

j=l,2,  ..N 


lim     2b 
k-^oo 


/-^j  (1  +  k  yr  -  ^2) 


=  0      . 


ftirtherrtore,     0  <  0^  <  0       so  that 


^   -   (^g/i^l)'^"'  .  1   -   (02/0,)''-^ 

1  -  ^fz'h^         -hT-7JJJ^ 


1  -0; 


k->^    1 


■^?7^ 


/     0 


Max      \V^{\^)\ 
Thus  for  0  <  b  <  ^,      lim      \\\<\ 

k-^oo    Max      |P  (\,b)| 
|X|<\      ^ 


2.     Let  b  <  b  <  1,    then 


Pk(^j.t)   =   (V2b   -  1")^ 


(^-) 


sin(k  -  1)0.        cos  k9 . 


sin  9 . 


^  + 
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cos  0     =     ■     J  _■  .     Hence, 
"^      /2b  -  1 

Maxjp  U,-^)! 

o<    1^1^^ 


^^1  +  k  JT^W) 


(2b   -  1)^/2         Max     3-      I  (^^  -   1)    sln(k  -  l)Q       cos  k9 


72b -1 


Again  since 


(^  <     y(2b    -  1)      for    %  <  b<T        ehL±±Jl~[Sl  ^  n  .  ^ 

(2b  -  1)^      ^°  ask-^o.. 


Nov 


lim 
k-^oo 


/-i-  _  1^  sin  (k-1)  Q   cos  kQ 
V  b    y   sin  0      +  -b 


does  not  exist,  and  hence 


Max  _ 
.cos9|<=^ 


/_!_  _  ;l^  sin(k-l)  Q   CO 
\^  b-    y   sin  9 


s  k9 


->  0  as  k 


Therefore  Tor's  <  b  <  1, 


lim  |\|sa" 


Max_  |P^(\,'^)| 


k-*""   Max_  |P  (X^b) 

I.I  j£ 


"^0   as  k->  "^^ 


|x|<\. 

2.9  A  combination  iterative  process 

As  was  noted  In  section  {s.6),   the  computed  ^^' s,   b^'s,  become 
constant  for  k  >  M.  For  this  reason  we  wish  to  consider  two  additional  pro. 
cesses  for  solving  linear  equations  In  this  section. 

Process  1   '>?   *M      ~* 

■"•  ^0'  7^""'7m   ^^^  generated  by  the  Chebyshev  iteration  scheme 

described  by  (2.24)  and  (2.20).   For  k  >  M  ^     ^ 

y   rui   K  ^  m,   >7j^^^,  7m+2*°-  ^^®  generated  by 

the  iteration  formula  (2.28)  with  b  ='^. 


^2=ess  2.  The  computation  scheme  Is  as  follows: 

Mp+q 

^^d   l3»4„  -,  =  1 

Mp+1      • 

Then  for  process  1,  after  (M  +  q)  iterations. 


X  - 


?M+q  =  ^q+iQ"   (x   -  fl)    -   (2b   -  l)   <^q'\^  .  ^.  ) 

q  (0 

Where  S^  and  Q  are  defined  as  for  (2. 30).     Then 


since 


^  ■  ?1  =  ^  -  ?M  =     ^^ihl^W^V^x  -  f^) 


and 


?  -  ^.  -  ;?  _^ 


<^0  =  ^  Ym-1  =  cosh(M  -  1)6^    '^^u.i^'h^  -  ^q) 


where 


/  ^IW  1  •   ■        ~        r>nc    IWQ  rv  ~1        .1 


=      cos  M9.,    9.    =   cos"     -«i 
J        J 

0     if     i  ?^  J  , 


X    - 


Hence, 

^to(q  .  1)9^   cos  M9^^         ^^^     Sin  49     oos(M  -  1)9. 
f^  sln9.  coshM«-p  "ilTg:    cosh(M  -  lja>      (2-36) 
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M    -M 
where  cosTi  Mc*>  =  ^ — ^  f  . 


We  vish  to  determine  the  maximum  of  (2.36). 


Now 


Max 
O<0<7T 


q  sin(q+l.)Q  cos  MQ     d+1  sin  q@   cos(M-1)0 
^   sin  0    cosh  M<^  "  r         


Max 
O<0<F 

Max 
0<9<F 

Max 
O<0<7r 


sin  0  cosh(M-l)6o' 

q/sin  q0  COS  0  ^  ^^^   ^  COSM0  _  0.1  sin^  cos(M-1)0 
^  V      ®  ^°s^  M6^   r    sin  0  cosh(M-l)^ 


h(M-l)6<.>, 


.cosh  M 


cos 


cosh  M^   cos 


:q+i 


cos  M0  -  P^"^^  cos(M-l)0  \  sin  q0    q  cos  q0  cos  M0 
'    cosh(M-l)^i  sin  0  ""  r    cosh  M^ 

f;"  ,  V  1  sin  q0  cos(M-l)0    q  cosq0  cosM0  -  sinQ0  sinM0 
h(M-lM       sin  0      ■"  e  coshMco 


-  I  cosh  Mo;  ■  cosh  {U-l)c4j      ^  "*" 


cosh  U(aJ 


since 


Thiis 


sin  q0 
sin  0 


£  q.   by  lemma  (A.- 2) 


/C- 


Theory  (2.5):   If  after  M  iterations  with  the  Chehyshev  process,  'b  is  used 
then 


.q+1 


cosh  May   cosh(M-l)6i> 


q  + 


cosh  UuJ 


^^ere    ^  =  ^  and  cosh  M^ 

1  +  4  -  X"^ 


-  ^^  ^  ^-" 


We  now  wish  to  compare  the  maximijm  error  of  process  1  with  the 
maximum  error  of  process  2  when  \\^    -  x| |  is  fixed. 

^eorem  (2.6):  If  after  every  M  iterations  with  the  Chebyshev  process  we 
begin  the  Chebyshev  iteration  scheme  over  again,  i.e.  process  2,   then  after 
after  Mp+q  iterations  the  maximum  error  of  process  2  is  greater  than  or  equal 
to  the  maximum  error  of  process. 1  for  | |^  -  x| |  fixed. 
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Pro2£-"   If  the  Chebyshev  sequence  is  rebegun  after 


every  M  iterations^  then 


ig 


)+q 


-^1 


t^^ 


< 


|Hf 


\V2 


u 


H 


MJ  (coshM^)P        cosh  qu>        • 

After  Mp+q  iterations  with  process  1,    then 


\l 


X 


H 


M+q  ■■      ^   /     •     -u 


\l/2 


A 


f 


,M(p-l)+q  ^M(p-l)+q+l 


.-l)+q) 


J 


^2^_^>^^    II  y       M(p-l)+q 

coshM^      -      cosh(M-lW    /r(p-l)+q^.    ^^^^^^ 


We  wish  to  show 


''^M(p-l)+q        J4(p-l)+q+l 


cosh  Mou     "  cosh(M-l)^      )(M(p-l)+q)  + 
or  multiplying  through  by  cosh  Mu) 


pM(p-l)+q 
cosh  Kui      — 


(coshM^)P   cosh  q^ 


M    -M 
Since  cosh  Mm.  =  ^  g  ^   ,  the  above  inequality  becomes 


)-l 


1 


Mco  cosh  qco' 


(.M(p-l)+q    M(p-l)+q+l  r'^1  +   (r^f 


M(p-l)+ql  +^M(p-l)+q 


<  .^ihsh^-^ 


2p- 


^(pFp  1 .  (f 2) 


or 


i.i^inif 


1 .  ((>^f-\ 


M(p-l)+q)  +  1  < 


(^^WP^T^ 


•    C2.37) 


(1) 


We  consider  two  cases, 
q  =  0. 
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2 
Let  jo  =  t,    then  we  wish  to  show 


1  +  t^   1  ,..    .  oP-1 


-— ^iM(p.)..,_^ 


1  .  t^-y  ^^^^  ^^  ^    -  A  .  .MW-l  •  (2.38) 

Now  for  i  =  0,  ._,...  M-1, 


t^-1  <  t^ 


M-1   .    ^    M 


and,  hence,       Mt*^""^  <  T.    t^  =  ^  -  '^ 


i=0     1  -  t 


so  that  Mt^-l(l  -  t)  +  t^  <  1  , 


Then  (1  +  t^)  ^  Mt^-l(l  -  t)  < 


or 


2 


..   .       [(1  +  t^)  +Mt^-^1  -  t)]P-l  <2P-1  . 
But  by  the  binomial  theorem, 

(1  -  t")5-l  .  „(p.l)t"-l(l  .  t)(l  .  t")P-2  <  [(1  ,  t«)  .  Mt«-^l  -  t)]P-l  <  sP-l 
IH-  M(p-l)t"  (J-  -  *)  <     g"'^ 

and  since  0  <  t  <  1 

1  .  M(p-l)t^  -11^  <  2^-^ 

1  +  t^-^  -  d  +  tV^'    •    • 

and  thus  we  see  that  (2,38)  holds  true, 

(2)  0  <  q  <  M  -  1  . 

Again  since  0  <  t  <  1, 
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+M-1  .  ,  1 

^         <  t       for  i  =  0,  .,  ...M-1 

and  t^*^"*"^  ^  ,  I+M-I  ^ 

^     ^  *      for  1  =  0,  1,  ...q  . 

Thus  adding  the  two  inequalities, 

t^-V  .  t^)  <  (1  .  .M-1^  ,i        (i  =  0,l,...,<M.l) 

or      qt^-^d  +  t^)  <  "^Z    (1  +  t^-1)  t^  =  (1  +  t^-h   1  -  t'^ 

i^O  ^  1  -  t 


which  implies  qt^-1  (l  -  t)  1  -  t^ 

1 .  t"-i  -  rrr^ 


Since     1+  t  -^(1  -  t)M(T)-l)   ^     2^"^ 

1  +  t^-l         -  (i^tV^ 


q 


But 


1 ,  t -V  -  t)M(p-i)     t;^:!^^^      ,p-i       , . , 

~      (1  -  t")P-i  (1  +  t^) 

2^ 


(1  -  t")P-^  (1  +  t*) 


and,  hence. 


^M-l[l  -  t][M(p.i)  +  q] 


1  + 1 .- . <  ^ 2 


P 


1  -   t^"'  -  (1  -  t^)P-^l  .  t^) 


and  thus  (2.3?)  holds  true. 
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2.10  The  vectors  r,  and  "z 
k k 


In  section  (2.2),  our  choice  of  polynomial  P^(g)  was  detemlned  by  our 
de.ire  to  minimize  |  |^^  -  1|  |  for  |  |,^.  -'5^1  (  fixed.  Let 


(0   ^ff  -Lxxea.  hez   us  now  consider  the 


effect  of  the  choice  of  polynomial  on  the  residual  vector  ?^  where  ?^  =  J  -  A;? 
and  on  t^   =  G^^  =  H^  -  ^^.  Again  we  shall  assume  H  is  positive  definite  and 


symmetric. 


since  (x  -  ^j^)  ,  Pj^(G)(J  .^^),  „,  ^^^^  ^^  premultlplyi,^  the 
atove  by  A  that 


But  since  \  =  ?  -  A^^  =  A(3  -  ^^),  (J  .  ^^)  .  ^'^  and  h.nce,  (£.39)  be- 


comes 


since 
From.  (2. 11), 


AGA-1  =  A(I  -  M)A-^  =  (I  -  AH)  =  H-^d  -  HA)H  =  H'^GH  . 


H"^GH  =  (F*)-lojloV  . 


and  therefore 

ti^-^th^;     i=i,2,..i*'^     -(i^j    i^i<^ 

As  was  shown  in  section  (2.4),  \  =  G^  +  H^  -  ^    =   (i  .  g)(x  -^.^) 

(I  -  G)(:r  -  J^)  =   (I  .  g)Pj^(g)(i  -  g)-1(i  -  g)(1  -  t) 


Max,  |p^(\)|  . 


Then 


and  hence,  f^  =  Pk^G)^^  . 


3h 


Consequently^ 


ll\IISlV°'iuli^0ll</]5~i=^..N"'^^'^^'  "'oil 
Thus  we  see  that  the  Chebyshev  polynomial  simultaneously  mini 


minimizes 


X  -  -  '  '  '  ' 


,5^,,    ^'^       Max   IK 

l='foll    11^ -foil  '    ll^oll    TW^ 


and  Max   ''\| 


^oll  ll^ol 


Max    N^k' 


In  [5],  Stiefel  chooses  the  Chebyshev  polynomials  to  minimize 

l^kll^^  11^0"   '   ^"^  ^^^"^  °^'^^^  ^^""^^   analysis,  the  iteration  scheme  of 
Stiefel  is  equivalent  to  the  Chebyshev  iterative  sequence.  His  choice  of  G 
is  G  =  I  -  oA. 

2.11  The  operator  G 

We  have  so  far  imposed  the  restrictions  that  H  be  positive 
definite  and  symmetric  and  that  the  proper  values  of  G  lie  between  -\  and 
X.   Two  operators  which  are  simply  related  to  the  matrix  of  coefficients  of 
the  original  equations  to  be  solved,  the  matrix  A,  and  which  will  be  shown 
to  satisfy  these  properties  are 

(1)  G  =  I  -  OA         (a  >  0) 

(2)  G  =  I  -  aD"-^A      (a  >  0) 


r 


11 


where 
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In  the  next  chapter,  it  will  be  shown  that  for  certain  matrices  the  second 
operator  affords  great  advantages.         : 

If  A?  =  vl>%    then  y    =     ^^  ,  and  since  A  is  positive  definite 

z  Dz 

^^  >  0.  Thus  the  proper  values  of  D'^A  and  A  are  positive. 

Let  C  be  a  matrix  whose  proper  values  are  positive  and  G  =  I  -  aC, 
Then  if  the  proper  values  of  C  are  0  <  a  <  v,  <  . . .  <  ^^  <  ^^  i,  i3  ^^^^ 
in  [2]  that  ^  =  -^    minimizes  the  largest  proper  value  in  magnitude  of 


G  =  I  -  aC.  If  a  = 


^  = 


a  +  b  '  "^^^^  ^^^   proper  values  of  G  are  distri- 


buted between  -/^-~^]  and  J^  '  ^ 


b  +  a 


b  +  a/* 


If  G  =  I  - 

polynomials  become 


a  +  b  ^'  then  the  iteration  formulas  with  Chebyshev 


'k+1 


2b,  ,   (I  -  - 
k+1  V     a 


^b^^^k^rTb^-^k.i^-^k.i  (k>.i) 


?i 


=  (I 


a  +  b  ^^fo  +  TT"b 


(2. 40. a) 
(2.40.b) 


Since  the  residual  vector  is  ?^  =  ?  -  a|,  the  above  becomes 


?k+l 


=  2b 


k+1 


a  +  b 


^k  ^  f  k  -  A-i] '  %., 


(k>  1) 


%  =  i-Tb  ^0  ^  f c 


(2.41) 


If  we  wish  to  solve  the  system  of  equat 


ions 
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where  A^  is  asynmetric,  then  as  was  pointed  out  in  section  (2.;l)^-  it  is 
possible  to  solve  the  equivalent  system 

where 

A  =  A%        and     ?  =  A*y,   . 


Although  a  large  number  of  elements  of  A^  may  have  been  zeros,  they  may  be 
destroyed  in  computing  A^A^.  This  can  be  avoided  for  the  operator  G  =  I  -  oA. 
Instead  of  performing  the  matrix  multiplication,  one  may  compute  ^    in 
equation  (2.4l)  by  first  computing 

and  then  computing  ?^  in  equation  (2.4l)  from 

This  corresponds  to  letting  G  =  I  -  aA*A  ,  and  H  =  oA*. 

2.12  A  comparison  of  the  methods 

Let  us  now  compare  the  number  of  iterations  that  are  necessary  so 


that 


If  we  use  the  iteration  scheme  described  by  (2.2)  with  G  =  G   then 


k' 


u  1    -k 


I 


We  wish  to  determine  k    so  that 


r  /\-\.Y'' 


or 


where 


^1     -"'  -  e 


\ 


■■m 


6 


Then, 


k     =  ^n  6  ' 
1       T—T- 

Xn  \ 

If,  however,  the  Chebyshev  iteration  scheme  is  used,  then 

Since  cosh  ko/  =  ^ ,  we  wish  to  determine  k^  so  that 

®  +  e  =     -—         and  thus     e     '^  =  ^ 

^-  i  +  ^rrTT^  • 


Consequently, 


A 


f 


k.  .,  1  -    ^1  ^    -'' 
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2  1  =  (2.42) 

1  +    (l-X^ 


5Q 


Then 


... /n  f 


1  +  v'l  -  6'^ 


For  the  iteration  process  described  by  (2.28),  if  b  =  ^  and  ^  =  Gf   +  nf 
then  by  theorem  (2.4) 

\\f     -t\\       /|h|   W^ 

llfo-3?||<(^THl7J     ^^i^ktTrF)    . 

Thus  we  wish  to  determine  k-  so  that 

(>   ^(1  +  k3  /l  -  X^)  =^'      . 

An  explicit  splution  for  this  equation  does  not  seem  possible.  Table  I 
.(page  41)  gives  some  sample  values  of  k^  for  \  near  1.  From  Table  I,  we 
note  that  as  £  decreases  k^/k^  increases.  We  now  show 
Theorem  (2.7):  Let 

^2 

ir  =  e(6',\) 


then 


for 


lim  9(6 ',\)  =  1 


0  <  X  <  1  . 
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^^^22£-   Nov  ^^^j-^^  is  the  maximum  of  the  Chebyshev  polynomial,  and  by 
theorem  (2.3)  ^^1  .  k  JTT^)   is  the  maximum  of  the  polynomial  associated 
with  the  constant  b  iteration  scheme  when  b  =  1)  and  ^^  =  g/  +  Hf.  Among 
all  k   degree  polynomials  which  satisfy  the  condition  P  (l)  =  1  the 
Chebyshev  polynomials  have  the  property  that  their  maximum  is  minimal  on 
the  interval  (-\,\).  Then  for  k  fixed, 


<  ^^(1  +  /l  -X^i 


cosh  k^  -  f  U  +  1^1  -  \  k)   . 
Since  kg  and  k  increase  monotonically  as  £'  decreases 


and,   hence. 


Since 


^2<^3 


0  <  9(6, \)  <  1        . 


^  ^(1  +    l^X^k   )   =  6' 


3' 


/n 


£' 


1   +/l    -   \^k. 


k-  =  ^ 3  ^ 

1  ^  \ll  -  x'^ 


and,    hence. 


^n               ^' 

^2 

.rn             ^_ 
1  +  1^1  - 

■^■^ 

^3 

^n              «' 

1  +  /l  -  \\ 


Now 


e  3(1 ,  k3  yrr^)  <  (.''3(1  ^  yrTF)''3 .  ^^3  ^ 

and  since  k^  and  k^  increase  monotonically  as  £-    dec 


jcreases 


k3<k^ 


Then 

Jn  ^ 


^  >         1  +  A 


l^Z. 


'3~   in 


/n  \ 


1  .  A  1  +    1^1   -  ^'^ 

. in 


1     - 


/n  >C 


/n  6' 
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TABLE  I 


\  =  .95 
6 


k^/k^ 


y^3 


10-^ 
U„9 


9.3 


11.9 


,21 


.78 


10"^ 

13^.6 


23.5 


28.5 


.17 


.82 


10-5 

224.5 

37»8 
kk.o 


.17 


.86 


\  =  .99 


kg/k. 


^2/^3 


229.1  687.3  1,1115,5 

.092  .078  ,075 

21-1  53.5  85.9 

'^^  .82  .86 

27. 3  65,0  100.2 


X.  =  .999 
k. 


^2/\ 


^2/^3 


230i.if  6,904.3  11,507.2 

•029  ,025  .024 

66.9  169.9  272.8 

.77  ,82  .86 

86.9  206.4  318.2 
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III   A  MODIFICATION  OF  THE  AVERAGING  METHODS 

3«1  Introduction 

In  the  previous  chapter,  ve  considered  several  iteration  pro^ 
cesses  for  solving  linear  equations  of  the  form 

Where  the  c^'s  vere  determined  hy  some  rule.  In  this  chapter,  ve  shall  con- 
sider the  above  iteration  process  for  matrices  which  are  symmetric,  positive 
definite  and  possess  property  (A)  (defined  below).   It  will  be  shown  that  by 
modifying  (3.I),  it  is  possible  to  substantially  reduce  the  number  of  iter- 
ations necessary  to  reduce  the  error  vector  to  a  given  value.  Furthemore, 
if  the  calculation  is  performed  on  a  high-speed  computer,  then  with  the 
modification,  the  memory  requirements  will  be  half  those  needed  for  (3.1). 

Definition  (3.I):  A  matrix  is  said  to  possess  property  (a)  if  there  exist 
two  disjoint  subsets  S  and  T  of  W,  the  first  N  integers,  such  that 
SvT  =  W  and  if  a.  .  ^  0  then  either  i  =  j  or  i.S  and  >T  or  i^T  and  jeS. 

If  a  matrix  has  property  (a),  then  it  is  possible  to  reorder  the 
equations  so  that  S  =  ^1,  2,  . . .  N^^  and  T  =  ^^N^^  1,  „..,  nJ.   m  this 
section,  we  shall  assume  that  the  equations  have  been  reordered  in  this 
fashion  so  that  A  appears  as  the  hypermatrix 


* 


where 


I?j_  is  an  N^  X  N^  diagonal  matrix, 
D^  is  an  Ng  -  N^  diagonal  matrix. 


and  S  is  N  X  N,  . 


Such  matrices  are  said  to  have  ordering  a- 


43. 


3o2  Proper  values  and  proper  vectors 

We  wish  to  investigate  the  proper  values  and  proper  vectors  of 
D-^A,   Since  D'^A  is  similar  to  ^^1^^-^)^-^!^  =  ^-^l^^'^l^  ^    ^^^  ^^^^^^ 
values  of  D'^A  and  jf^l^Klf^l^   are  the  same.   Furthermore,  if 

vz,  L>  AD    z  =  VD  /  z,  and,  hence,  the  proper  vectors  of 
D-^A  and  D'^^^-Vs  ^^^  related.   The  vectors  which  will  be  considered 
will  consist  of  two  subvectors  so  that 


where   v 


'1 


and  ^(2) 


Theorem  (3,l):   If  D"^/^^"^^  ^^^ 


D-V2ad-V2/| 


(1) 
(27 


property  (A)  and  ordering  <r      and  if 
-(1) 


(1  +  n) 


W 


then 


D-V2;^-V2  {-A 


(1) 

7(27i  =    ^1  -  ^^) 


for  [1^0, 


Proof:  Now 


d-V£ad-V£  . 


D-V£sD-l/2 


D, 


D-VVd-1/2 


hk 


Where  I^  and  I^  are  N^  x  N^  and  N^  x  N^  identity  matrices,  respectively. 

Let  d:^/Vd:^/2  ^  ^*  ^^  ^^^^  ^  .^^^ 


(1  +  n) 


T*|(2)  ^  ^|(1)  ^^  ^(1)  ^  ^^(2) 


Then 


-d)  .  .M) 


=  (1  -  n) 

-1/2  -1/2 
Thus  D  /  AD  /   has  (say)  2r  proper  values  1  +  u  1  +  n        i  .  „ 

and  N-  -  2r  proper  values  equal  to  one. 

The  proper  vectors  of  (D-^/^AD'^^^   -   j)  are  the  same  as 
(D-1/2aD-1/2),   Since  D'^^^AD-^/^   has  proper  values  1  +  ^^,  1  +  ^,  , 
-'A  t^-^    i^^l^  0)   a.ndl   repeated  N  -  2r  times,  (d'-^/^AD"^/^  -  I)  has 
proper  values  i^^,  -^^^,  .-»,  i^^  and  (n  -  2r)  proper  values  equal  to 
zero.   Let  us  assume  at  least  one  element  of  T  is  non-zero.  Then  there 
exist  no  non-zero  linear  combination  of  the  row  vectors  of  (o|T*)  equal 
to  a  row  vector  of  (t|o)  and,  hence, 

rank(D-l/2^-l/2  .  j)  ^  rank(o|T*)  +  rank  (t|o) 

=  2  rank  T 
since  rank  T  =  rank  T*,   If  all  the  elements  of  T  are  zero,  then. 


h3 


rank  (d'  '  AD""*"' ^  -  I)  _  P  ^.^.^y,   t  _  n  ^«  4.^.  a.  * 

^   "^  ^^"^  ^  =^  so  that  r  =  ran]?  T.  Now  T  is 

\  X   N^  and  T  is  N^  X  N^;  thus  r  <  min(N^,N  ), 

Therefore  T*J<2)  ^  -  ^^^  ^^^  _  ^^  ^^^^^^^^  independent  solutions, 
and  T?    p  -3  has  (n^  -  r)  linearly  independent  solutions.  By  the  Gram- 
SchmKit  orthogonalization  process  the  vectors  t[^}   ...,  ^^2)^  ^^^  ^^  ^^^^ 
orthogpnal  to  one  another.  Since  the  new  vectors  are  linear'^combinations 
Of  the  original  set,  they  too  satisfy  the  homogeneous  equations  T*!^^)  ^  ^^ 

Ifaturally,  this  same  process  can  be  applied  to  3^^\    ...,  ^d)   .  ^hua 

N^-r 


IM.  -r 
we  assume  in  our  future  discussion,  the  vectors  f  5^2)      ^^2)  ^ 

/  ^1  '  "^"^  %  -r  > 
are  orthogonal  to  one  another,  and  likewise  il)^^^      ^(l)  )  ^  ^ 

Then  the  spt.  nf  Tro/^+,^v,<-  •'^'       -^ 


^  •  '^ 


are  If  -  Sr  proper  vectors  of  D'VSad-I/^  ^^  „^  ^^^^^^^  orthogonal. 

Therefore, 


i  =  1,.=,,  r 
i  =  1.  ...,  r 


D-V2^-V2J^  =  5^ 


i  =  1,  .,.,  Nj^-r 


i  =  1,  ...;  Ng-r 


V 


(3.2) 


The  proper  vectors  will  also  be  considered  orthononnal. 


V 


kS 


Theorem  (3-2):   For  the  set  of  vectors  described  Tby  (3.2), 


=  0  . 


anci  lor  every  proper  vector  g  7^  f.^  S  ^  e. 
Proof;   By  theorem  (3,l), 

.  [IF 

S^nce  e^  and  f .  are  orthonormal  vectors, 
Therefore,    [^i[^\   S^^^l  =  f^^',  4^^^  =  1/2  . 

3f3  The  modification 

In  chapter  II,  it  was  noted  that  for  G  =  I  ~  aD~^A  the  magnitude 

(Jf  the  largest  proper  value  of  G  is  minimized  when  a  =  a  =  -^-~      wh^^^  = 

a  +  b   *""^-^«  d. 
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is  the  smallest  proper  value  of  D"^A  a^.d  b  is  the  largest  proper  value  of 
D  A.   By  theorem  (3.I),  if  1  f  ^  is  the  largest  proper  value  of  D'^A, 
then  1  -  ^i  is  also  a  proper  value  and  hence  must  be  the  smallest  prope:e 
value  of  D"  A.  Then  for 

cc  =  'a  =  1, 
the  magnitude  of  the  largest  proper  value  of  I  -  aD'^A  is  minimized  irlth 
respect  to  a. 
Thus 


(3.3) 

and  the  largest  proper  value  of  G  is  ]!.   Let  us  now  consider  the  iteration 
formula  (3,1).  Substituting  (3.3)  into  (3.I),  we  have 


=  2c 


k+1 


D 


-1 


1  0  /  fk-l 


^2]  t(2) 


Expanding  the  hypermatrices,  we  have 


.(1) 


k+1   ^k+1  1 


=  2c^^,D,    r  '  -  S 


-  hill"^  ^  '^'^' 


.fk-i)*  K-i 


If  in  addition,  t     =  Gf    +  Ef 


(3.i^.a) 
(3A.b) 


(1)  _  T,-l/t,(l)   c*>?(2) 

0  J 


h'^^hTr-'-s^ 


(3. 5. a) 
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?f  ^  =  ^;'( 


i 


<t(2) 


-  S: 


:i) 


(3.5.-b) 


If  the  iteration  process  described  by  (3.I)  is  used  on  a  high-speed  computer, 

then  to  compute:  ^^_^^  it  is  necessary  to  have  ^^   and  ^   .  Thus  2N  memory 

locations  are  needed  for  storage.  For  matrices  under  study  in  this  chapter, 

however,  we  are  able  to  take  advantage  of  the  large  block  of  zeros. 

With  ^Q   and  ^  %  we  are  able  to  compute  ^g  by  {3,k.a.), 

Furthermore,  as  we  compute  each  component  of  ^^  \   we  may  replace  the 

corresponding  components  of  ^^^^  by  it„  With  ^^^  and  ^^^^,  we  are  able 

to  compute  f^^^  by  (S.i^-.b).  And  in  general,  ^ith^^^^^  ^^^/gk-l^  '^^  ^^^ 

able  to  compute  ^^^^  by  (3.^. a);  ^"^^  ?2k-l  ^^^  fik^ '   ^^  ^^^  ^^^^  ^° 

(2) 
compute  ^2k+l  "^^  (3.^.b).   If  f^   =  G^^  +  Ef   for  the  original  process,  with 

^Q   we  are  able  to  compute  p^'^   by  (3.5.b)  in  the  new  process.  Th\is  after 

k  iterations  we  have  computed  the  vector 


^^.^   at  no  time  has  more  than  N  memory  locations  been  necessary.  We  do  not 
have  the  vector  ^^,  and  we  are  unable  to  compute  it.   Note  that  at  each 
stage  the  latest  iterate  is  used. 

Our  iteration  scheme  now  becomes 


f; 


2k 


^2k+l/ 


%a' 


0 


0 


"2k+l°2 


aaid 


Let 


r  =  <CV^^  -  3^(^))  . 


OS  /  0  I  0 


=  li,   f-i 1  =  L,   I    %—    I  =D 


0  10/         I  S  I  0  y      \  0  I  D 


and      /^2k-'-l 


0 


0 


°2k+1^2 


—  =  C^f      then  L  +  D  +  U  =  A 


and  (3-6)  becomes 

■    %   -  2V-l(y'  -  U/^.^  .  Ij;  .  Dj|_^)  .  f^_^.  (3.,) 

If  Cj^  =  bl^  b  a  scalar,  then  (3.?)  describes  the  successive  over- 
relaxation  method  of  D.  Young  [9].   If^  however, 


■^   ^   °  I  ^2k*1^2 


where 


and 


2  -  ^t^ 


b^  =  l 


A'^  =B-^(y(2).s;(^)     , 


then 


/2k+l 
is  the  k""  iterate  of  the  modified  Chebyshev  iteration  scheme. 


k9 


where 


and 
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Equation  (3.?)^  then  may  he  rewritten  as 

Vl  =  ^°  -^  ^V^'^  f^  -  2C^(U  +  D)] 


^^'^  ?k  =  Jk-1  ^Pli^^  jL  =  ^  i^  ^^*  iHk.il  ?^  0.  We  shall  now  show  that 
this  condition  is  satisfied  for  successive  overrelaxation  when  b  >  0  and 
for  the  modified  Chebyshev  method.  Since  A  is  positive  definite,  the 
elements  of  D  are  positive.  Then  since  2Cj^L  is  a  lower  triangular  matrix, 
det  JD  +  2Cj^l|  =  det|D|  >  0.   For  successive  overrelaxation  with  b  >  0 
det|bl|  >  0  and,  hence  ^^^  >  0.  For  the  modified  Chebyshev  since 
\>^  >   1/2  by  theorem  (2.1), 


In  addition. 


det  f  ^-^4- — 1  >  0 

2k+l  2 


^k-1  ^  \-l^  =   I 


so  that  ]  |J^_^  -J^\  I  =  0  if  and  only  if  J^  =  5.  Consequently,  we  may 
terminate  the  successive  overrelaxation  method  and  the  modified  Chebyshev 
iteration  scheme  when  |  |J^  -  J^_^| |  is  less  than  some  preassigned  number 
and  be  assured  that  the  error  vector  is  related  to  this  number. 

3.^  Bounds  for  the  error  vector 

In  chapter  II,  we  were  able  to  describe  an  upper  bound  for  the 
length  of  the  error  vector  1  -  ^^.      We  desire  now  to  describe  an  upper 
bound  for  the  length  of  the  vector  x  -  ?  . 
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By  (2.9).  (^^  -  $)  .  P^(a)(|^  -  ^)  =  p^(j  .  :)-i^)(^^  .  .). 

We  now  expand  (/^  -  1)   ±n   the  proper  vectors  of  I  -  d'^A  which  are 

given  in  section  (3.2).   This  is  possible  since  the  elementary 

, ,  -1 

divisors  of  I  -  D  A  are  simple.  Thus 


N,  -r         N-r 


Now 


(;,  -  t)   =  P^d  -  ^-hKf^   .-)  .  D-V2  fta,P^(,^)e;  .  i^ 


a.   P  (-U. )f 
i+r  k^  ^i'^  1 


-^  P^(0) 


N,  -r  N  -r       ~l 

Zj  a^   .  P .  +  Z  a      1? 
.^^  2r+i^i   .^^  ^N^fr+i\ 


so  that  from  theorem  (3.1) 

It  will  be  shown  that  the  polynomials  we  wish  to  consider  have  the  property 
that 

P^(-..)  =  (-l)'^Pj^(^)  (3.8) 

and,  hence, 

P,  (0)  =0  for  k  odd.  .    , 

Then  by  (3.8)  and  (3-9),  we  have 

N. -r 

-..oPoJ0)l?l) 


;^^^^^^=^^/^^|(^i-^i.^^ 


j_  i+2r  2k' ^-^i 
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Let 
;(i^'  -  S<^)  .  4^)    ana   ^<^i,  -  J(^)  =  4^)  , 
then  by  theorem  (3.2) 


(4^''  V?)  -  \   i  (a,  -  Vr)^  4(^.)  ^  '^r  4.r4(0) 


r  N^-r 

E  (a.  -  a   )^  P^  fLL  )  +  y 
i=l  ^    i+r^   2k^^i^  -^  .^^  -i+2r'2k- 

and 


l2  ^2 


(,^k  ^  °2\y  =-^  Z  (a.  -  a    )^  P^    (n  ) 
Hence ^ 


42r4(°) 


From  this  we  have 

(3-10) 
Nov 


^i=l  ^    1+^   1    1^1  i+2r^i 


^0  '=  ;6 '-  ^ 


so  that  by  theorem  (3.2) 


0   ^  Vo  J  =  2-,>.(^  ~  Vr^   +  ^,  ^+2r 


Th-.JG  (3.10)  becomes  , 


0,|i^..M^ 
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or 


\\:>'''%\f<  ns.  /4^;;774j7iiDV|Wir.'   (3.11) 

Using  relation  (A.IO),  we  have  now 

Consequently,  we  have 

Theorem  (3.3)  ^   If  T  is  computed  by  (3.7)  and  if 

(1)  G  =  I  -  D'^A  , 

(2)  Pj^(-fi)  =  (-l)\(n) 


then 


l^.il   ,/lVu^'/' 


lljrr^-I^Ti^^ 


Max   ./p^,  (u.)  +  p2   (n  ) 


1»   O  f-*, 
r 


Now  the  unmodified  methods  converge  if  and  only  if  Max    |P  (w  )!-♦  o 

Tn   "j 
as  m-*<=o  ,   Then  since  ,°'%'**^r 


Max   /P^^(,^).P^^^^(,^)<   Max    iP2,(.,)|.   Max    \P^^^^i^^\ 

we  have 

Corollary  (3-31):   Under  the  hypothesis  of  theorem  (3.3);  if 


Max   |P  (ia.)|  -»  0  as 
'  mi  ' 


m  — >  e«3 


then 


0,n^..^i^ 


,->(l),,   -^  0   as   k 


:^o 


■   if  in  (3.1)/  Cj^^j    =  1/2  then 

/lc+1  "^  %-'-  ^^     • 

Thus  P,  (G)  =  G  ,   and.   hence,    condition  (-2)   iq  qnt-' q-p-,- ^^       mv,^^ 

K  '  f  )    ^^iiu.j. uxuii  v^;   IS  sax.vfariedo     This  process  corresponds 

to  the  Gauss-Seidel  method.     Then 
and,   hence. 


0^  /4(-)  -  4.i(^)  -  i^'  ^^  . 


Corollary  (3.32): 

IfPj^(G) 

=  G^,    then 

?J 

For  the  Chehyshev  iterative  process 


p  (n)  =  COS__k©_ 

V^^   cosh  k«^    ^ 


Where  ©  =  cos"^  -H_  and  -^  =  cosh'^c^.  This  again  satisfies  condition  (2). 
Purthemore,  the  maximum  occurs  for  P^^i^^)   and  l?^^^^i^^)   when  ^^  =  ^.  Thus 
we  have 

Corollary  (3.33):  If 

k^^''       cosh  k«*>   ' 
then 


11^^"  <  f h!u ) '^' /ZTiTziiz: 

^0      "         V'^U/  /     cosh^  2k*^      cosh^(2k+l)60 
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If  we  modify  (3.I),  vhen  c^^^  =^  and  with  f^  ^   G^^  +  H^  then  we  have 
a  specialization  of  successive  over relaxation,   For  this  process, 

P  (u)  =  P^     (-^  -   1^   sin(k  -  1)Q   cos  kQ 


cos  9  = 


and 

^    i./TT^     • 

This  again  satisfies  the  relation  that  P^(-^,)  =  (-l)^^(^)  .   Furthermore,  the 
maximum  of  P^^{^)   and  Pgk+l^^^^  °==^^  ^^^^  M-  =  H-   Thus  ve  have,, 
Corollary  (3^3^):   If  the  successive  overrelaxation  method  is  used  with 


then 


Let  us  also  consider  the  processes  which  have  been  termed  process  1  end 
process  2  in  section  (2.9).   The  polynomial  associated  with  process  1  is 


O^) 


q   sin(q  +   1)9      cos  M9_  _      q+1   sin  q9   cos(M   -   l)9 
'  sin  9  cosh  M^  "   ^  sin  9  cosh(M  -  l)^ 


where  cos  9  = 


-^     .      By  theorem  (2o5) 


Max    |Qi-^^(u)|    =  / ^ 

0<|j.<il     ^+^        '        (  cosh  Mij       cosh 


U-  l)a,)^     -^ 


cosh  M«; 


ill^^y 
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Again  <<-.)  =  (-l)"^^4l)(.) 

Corollary  (3^35):   If 


and,  hence,, 


then 


P   (it)   -   ^^     sln(q  +  1)q  cos  M9 


'M+q^ 


f 


sin  9 


cosh  Muf 


q+1  sin  qQ  cos(M  -  1)Q 
^    sin  9   cosh(M  -  l^j  ' 


0 


D, 


< 


I'u 


1/2 


D 


u 


4i^(^) 


1  2 


4kii(^) 


Associated  with  the  process  2  is  the  polynomial 

0(2)  /  \  ^  /cos  M9_  \^  cos  (^9 
^+q^'     I  cosh  Ui^  J        cosh  qc^ 

where  cos  9  =  -^  .  By  theorem  (2.6) 


^^  '€.(^)l  = 


(cosh  M^)-^ 


(2) 


cosh  q_u>  T4p+q^^' 


Also 

so  that 
Corollary  (3.36) 

then 


If 


4p],(?) 


Mp+q^^^ 


cos  M9 


cos   q9 


cosh  Uu)j         cosh  q6^ 


\ll     JI^Lu 


1/2 


ll^ 


u 


[ 


il\^) 


[illr.)] 


3.^  Comparison  between  various  iteration  schemes 


3.51  Comparison  of  various  modified  methods 


In  chapter  II,  we  saw  that  after  k(>  2)  iterations,  for  \\^     -  t\\    fixed, 
the  maximum  error  of  the  Chebyshev  iteration  scheme  was  less  than  the  maximum  error 
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of  the  constant  parameter  iteration  scheme  when  b  =  "&  and  ;^  =  G/  +  H^.  The 
question  arises  whether  after  k  iterations  for  [  |^^^^  -  x^^^\\    fixed,  the  maxi- 
mum error  of  the  modified  Chebyshev  method  is  less  than  the  maximum  error  of 
the  .successive  overrelaxation  method  when  b  =  'b  and  |^^^  =  D""^/y^^^  -  S?^^-^). 
Theory  (3.4):  For  |  H^^^  |  |  and  k  fixed,  the  maximum  error  for  the  modified 
Chebyshev  iteration  scheme  is  less  than  the  maximum  error  for  the  successive 
overrelaxation  method  when  ^[^"^  =  'D'J'  f  }^^^   -  si^^A     and  b  =^. 

^^2^'     ^^"^     cosh  mcj    ^^  ^^-^   maximum  of  the  Chebyshev  polynomial  and 
p  (1  +  m  4  -  jl  )  is  the  maximum  of  the  polynomial  associated  with  the  one  para- 
meter iteration  scheme  when  b  =  ^  and  f ^  =  G^^  +  By'.     Among  all  k*^  degree  poly- 
nomials which  satisfy  the  condition  F^{l)   =  1,  the  Chebyshev  polynomials  have 
the  property  that  their  maximum  is  minimal  on  the  interval  (-jT,  jl) ,  Furthermore, 
the  Chebyshev  polynomials  are  the  unique  set  of  polynomials  which  have  this 
property.  Thus 

^^  <  f^\l  +   2k  /iT^)  .  (k  >  0) 


cosh 


and    —  , 
cosh 


T2k\  1)^  <  e^^^^^l  +  (2k  +  1)  /l  -  ^)  (k  >  0) 

and  hence. 


1  , 1 

2  '^   p < 

cosh  2kw   cosh  (2k  +  1)(aJ 


l[^^\l  .   2k  ^^\[f'^-'\l.   (2k.l)  ^ 


Then  from  corollaries  (3.33)  and  (3.3k),   we  see  the  maximum  error  for  the  modified 

Chebyshev  iteration  scheme  is  less  than  the  successive  overrelaxation  method  when 
h=S,  and  |(2).D-1(?(2).  3^(1))   _ 

We  now  wish  to  consider  the  number  of  iterations  necessary  so  that 
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lljl-2|| 


■^*(i)   Tuxr  ~  ^ 


If  we  use  the  Gauss -Seidel  method^  then  by  corollary  (3.32) 


We  wish  to  determine  k|  so  that 


I'u      —     1    L       —2 
id] —  I  ^         VI  +  la^  =  <^  . 


Then  /|dL  \/^ 


Vu/    TTTW 


2  /n  n 
Next;,  we  wish  to  deterxuine  the  number  of  iterrLtlons  recessar  -  so  tho/ 


0 


IJ.  -X 


for  the  modified  Chebyshev  method.   By  corollary  (3.32), 


1 1^(1)    -?(i),,   -    Idt;    /       J       2 —  "^  — 2, T" 

I  IJo   -^11     \  '  '/  /      V  cosh^  2ka;   cosh^(2.k  +  1)^ 
We  wish  to  determine  k'  so  that 


'  I'u  1 


D 


1  1, 


,   /  2       ? 

-^  /     \'  cosh  2k^6^   cosh  (2k'  +  l)uj 
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Since 


K nh^\'  j\\K 


^'^       /IDJ       \V2 


=oBh(2k..i).l^  T51J7     ^  [TBip  /      /  ;;;;t 


cosh'-  2k^w      cosh^(2k;+l)^ 


cosh  2k' ct;  I    l°|/ 


we  my  solve  the  right-hand  side  and  left-hand  side  of  the  above  Inequality  to 
get  upper  and  lower  bounds  for  k'.  The  calculations  are  the  san,e  as  for  deriving 


(2.42). 

let 


,   .  >]^f"       . 


t  I 


D. 


i'u/    /i" 


then 


where 


(3.12) 


-2 
-  1^ 


^'-TTt 


From  the  above  equation,  we  get  an  estimate  of  k'  which  Is  in  error  by  at  most  1/2. 


We  shall  later  compare  k  with  k ' , 


For  the  successive  overrelaxation  method  with  b  =  "B  and 
^(2)_  -1  A(2)   ^^{1)\ 
(1  "  2  (^    ~     fo     )  >    "^^   ^^^®  V  corollary  (3,34) 
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Thus  ve  wish  to  detenjiine  k'  so  that 

|B  I  V^^       2k'   / -^ — ■ 

^  ^  (3.13) 

An  explicit  solution  of  this  equ.ation  does  not  seem  possible.  However,  we  have 
the  following. 


Theorem  (3.5):  Let 


^2 
^3 


Then  lim  9 '(6,|i)  =  1 

for  0  <  ^'  <  1. 

P3-oof:  By  theorem  (3-^);  for  |  |Iq  ^|  |  and  k  fixed,  the  maximum  error  of  the 
modified  Chebyshev  iteration  scheme  is  less  than  the  maximum  error  for  the 
successive  overrelaxation  method  when  f\      =   D"^  (y         -S^^^^)  and  b  ="%, 
Then  since  k^  and  k'  increase  monotonically  as  £  decreases 

k,'  <  k 


-  ^^3  ' 


and,  hence, 

9  <  9'(6,^)  <  1 

From  (3,13),  we  see 


/n 


S— )   '   ^.2k.yi-^f  .(.2^l.(2k..l)/rT=^ 


An  c 
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and,  hence,  'pf  (3.12) 


2 

^3 


/n     I        £. 


1 1 


^/n 


f 


/n 


I'u 


W 


y 


("i  +  2k'  JTT^f  ■"  f ^  f^  -^  (2k'  +  %  /TTpJ 


Nov 


SJc 


f  3  /^i ,  2k'  K^y .  p Y^ .  (2k^.i)  iT^Y 

.      ^^3   iY       r-~^VS        T,       / 5A^     /   ^^3      -^^3 

<  f  y  (i  +  /i  -  ;r2  j   3  -.  p^^i  +  yi  - 1)  ^    =/  ii     -^  ^ 


where 


f  ' 


1  +  yi  -  U 

and  since  kj^  and  k'  increase  monotonically  as  £  decreases 

kJj     <     kj_     . 


Then 


^^-kT^ 


/n 


1  + 


/TTTT^      g 


1,2 


f 


3  A 


/-|D|-     \l/'2 


fe;4'  '/f'  ^  ^''i  ^'  ^  P'6  ^  '^-^i^^)  ^ 


/n^S^ 


'm 


1  - 


m 
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Since 


'n 


jDj 

ID" 


1/2 


l'u> 


ki  = 


/T 


+  [1 


2/n  ji 


it  is  not  diffic-ult  to  verify  that 


lim 
^0 


/ 


=  0 


n  <f 


and  hence 


lim   "^2 


£-^►0   k^   -  ^  - 

Theoren  (3.6):   For  k  and  | ||^^^ | |  fixed,  the  maximum  error  for  the  modified 
process  1  is  less  than  or  equal  to  the  maximum  error  of  the  modified  process  2 
described  in  section  (2.9). 

Proof:   By  corollary  (3-3^),  the  maximum  error  for  the  modified  process  1  is 
after  k  iterations  when  |  |^i   | |  is  fixed 


D 


I'u 


1/2 


m 


A 


i^\^)  \' . 


^^IL  (^)' 


and  the  maximum  error  for  the  modified  process  2  is  after  k  iterations  when 


1^(1)11    .      ^ 
\6q    'II    is   fixed 


(ptj  '/S^ 


ill  (^)' 


However,    by  theorem  (2.5) 


-ilh-^}y<l^il\-.)J  and     [^JJlj 


<v 


Qi!:',  G)J 


^2k+l 
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Hence 


'E^^^n^isw 


< 


QAr/  (n)J'  + 


(2)  ,^^  |2 
2k 


3«  52  Comparison  between  the  modified  and  unmodified  methods 

From  equation  (2A2)  for  the  unmodified  Chehyshev  iteration  schei 


me 


■^2  = 


Jxy  p 


where 


^        '  TdT 

'  'u 


1 1/2 


For  the  modified  Chebyshev  iteration  scheme,  we  have  by  (3.12) 


A 


1  +\/l  -  £"^      1 


A 


Jn    -2 


2  <k'< 


1  +  \/l  -  £"2 


/n^2 


where 


D' 


/ 


1 1/2 


Then 


and 


(i^i'uy     /? 


£'     <      6"    if   2|dJ   <   IdI 
-   '-  '  I'u  -  '  'u 


£'   >  6"    if   2|dJ    >   JDl 

'I'u     '  'u 


Hence  if  2|D^|^<  |d|^  and  ^'  <  1  , 

kj 


-2_  <  _1. 
k^       2 


We  will  now  show 


6k 


lim 
6-*0 


•=2 


(3.11*) 


Now 


_1_ 
2 


L^ 


Jn 


^n 


1   +    \ll    -  6-'.'^ 
6' 

1  +  /l  -  ^'^ 


>! 


r 


/n 


1  +  /I 


^i^ 


2  ^  _^    1  +  /i  -  e""^ 
2       xfn 


^f: 


1  +  /I 


a  = 


b  = 


u 


D 


u 


Then 


A 


ae 


lim 


and  thus 


i  +  /rT^2  2 


a  ^ 


1  +  /l  -  h'k 


lim 


1  + 


/n 


a 


^n  e 


/n  b 

1  +  Tp — 


Yn  1  -Hl/T 

2   2 
~  a  e 

n   £ 

jni^  /r~- 

^'e' 

Jn 


-  1 


^2 


_1_ 
2 


6^-?  0 


We  now  wish  to  compare  the  one  parameter  iteration  scheme  when 
^1  "  %  "^  ^^  ^^^  b  =  b  with  the  successive  overrelaxation  method  when 


f[^^ :  .;X^/^^  -  s;(^)J 


We  will  now  show 


lim  3_ 


_1_ 
2 


Since 


■i 


■^2 
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K         i.-m  K       ^.^      K       .,._  ^2 


lim   ^3  ^  llm   3    lim  _f2_   lim 
6-^0       k^     ^-^0     k^   *  «f-^0   k^ "  '  6-^0     ^3 

and  by  theorems  (2.7)  and  (3-5),  and  (3.1U) 

^-^0  ^3         2  2        • 

3.6  Bo-unds  for  the  error  vector  for  successive  overrelaxation 

In  section  (3.^),  we  developed  bounds  for  the  successive  overrelaxation 

error  vector  under  the  assumption  p[^^  =     D'Yy^^^  -^fo^^)   '      ^^   ^^is  section, 

we  shall  arrive  at  a  bound  independently  of  this  assumption. 

As  in  section  (3A),  let  us  expand  (^^  -  t}   and  (^^  -  t)   in  tenns  of 

the  proper  vectors  of  G  =  I  -  D~^A.  Then 

^°  -  '= °"''  [k  v^"^ '  A  v.A  ^  1\,.3A  ^  X  %.-.-^^) 


and 

Now  by  (2.30), 

,-l/::> 


N^-r  Ng-r 


■'^^-  /i=l  ^^        i=l  l^^^^i   i^l  1.  i+2rPi  -^  ^^^  ^i+N^+r^iJ- 


^  -  ^  =  QS^Q-  (f^  -  t)    -  (2^  -  1)  QS^_,Q-1(^Q  -  5) 


so  that 


N^-r 
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and 


Ng-r 


where 


Then 


^  T   sin  m9. 
m-1 


m^  i'   r     sin  9.     ii  w^  jt  u. 

m-1 
=  mp  if  9^  =  0 

/  , \m-l  m-l 
=  (-1)   mp       if  9.  =  TT 

and  =  /2^  -  1  =  £= 

1  +  /i  -  ;i^ 


and  hence  S^(0)  =  0  for  m  even. 

Then  in  view  of  the  above,  we  have 


N  -r 


-(^^-^)^%i.2Al^l(0)pi- 
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and 

:>(2)        -»'(g)  1/2  )'^ 

+1     a,    .    „       s  (0)0^^^ 


llBf  4^'||^  =  -^  Jj(a^  .  ^l.r'S^.f'^i)   -  (^  -  1)(V  -  a,^,^^)S^^,^(,^)] 


2 


N  -r 
k2 


H-  (^  -  ir     E      a2       ^  S2       (0) 


and 


,.1/2^(2)1  |2         1      ^.  (., 


Ng-r 


Now 


and 

s6  that  by  theorem  (3.2)  "^ 

K[  -r 

and 

N  -r 
Mb^/2^(2),.i2         1       V  ,  x2         I       2 

'20  ^     1=1     ^         l,x+3r'  .^^     0,i+N^+r 
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Then 


||DV2/j|2n|Dp4^^||^.||DV2,f)|i2 


+  (2b  -  1)^  Z  af     S^   (0)  +   Z  a^      q^   Cn^ 
i=l  0,i+2r''2k-l^"^  +  ^t*j_  ^l,l+N^+Ak+l^°^ 


and,  hence  by  (A.l) 


H-(2S-1)    Max   /s^^^,(^)  +  S2^(^x)    |iDV2;(l)||   , 


0<ii<n 
The  maximum  of  S  (|j,)  Is  attained  at  (j.  =  jlj,  and  hence 


m 


1^.11  i/4(?)-Li(i^)H^rikrii 


y 


and  thus  by  the  Cauchy-Schwarz  inequality 


i  /(4(S)  -  4.i(^))({^  -  (2^  -  i)X4-i(^'  -  4(^) 


•  /T^Fii^^Mit^^ 


\ 


'^i( 


0 


^ 


"^eorem   (3-7):  For  the  successive  overrelaxatlGn  method  with  b  = 'J; 


l?JI        /.  _       „        .In 


where     •' 

Then  hy  comparing  the  results  of  theorem  (3.6)  with  corollary  (3.34) 

that  a  cancellation  of  terms  takes  place  when  4^^   =   D'Vy^^-^  -  S$^-^^')      in 

the  same  fashion  as  for  the  one  parameter  iteration  scheme  when  ^  =  ^t.  +   Hy 


we  see 


and  b  =  b. 


3.7  Bounds  for  the  difference  vector 

As  was  pointed  out  in  section  (3.3).  a  possible  termination  procedure 
for  the  successive  overrelaxation  method  and  modified  Chebyshev  iteration  scheme 
is  to  stop  iterating  when  |  \J^   -  j?^_J  |  is  less  than  a  pre-assigned  number.  We 
shall  now  develop  for  i  \f^  -  f^J\   bounds  which  will  be  used  subsequently. 

Let  us  assume  as  we  did  for  theorem  (3-3)  that  G  =  I  -  D"\  and  that 
\(-i^   =  (-1)  Pl5.(^^)•  Then  we  have  from  the  proof  of  theorem  (3.3)  that 


/  1=1  -L_3_        "^^ 


t(2)_^(2)  _^(2)    ^(2)    T^-l/2y?, 


.   )P    (n  )t^^^ 
i+r^  2k+l^^i^  1 


Hence,  We  have 


f'    -P  ^P    -p..  -   Bl^/^f  J^(^  -  ^„)tW^l)-...3(^l))f ' 


W^-r 


*  ,^,  ^.2r[f2k<°)  -  P2k-2(°»P^^'* 
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p{2)_  M2)    ^  ^{2)      _^(2)      _   ^-1/2)' 


Then  by  theorem  (3.2), 


N  -r 


+   (P2k.l(^l)    -  ^Slc-l^^l?)" 


^  ,^,     ^-Srt^Sk^O)    -  ^2^-2(0)]^ 


(3.15) 


From  this,    we  ijave 


■''^'(JK-Zk-l^llS 


Max 


(^2k(^l)    -  ^2k-2(^lf  ^  (P2K.l(^..)    -  ^2I..l^^i)) 


; 


2 


r 

Z     (a, 
1=1       ' 


a.      )' 


N^-r 

Z 

1=1 


2 
a 
l+2r 


Max 


1       "^0 


Then  using  (A.IO),    we  have 


!ljk   -  jl-lll   ^     .'^'^         /tP2k(^l'    -  P2k-2(^)J'  -    [P2k.l(^l)    -  ^2,-1(^1)1' 


(3.16) 


For  the  modified  Chebyshev  method, 


cos  mO 


m      cosh  m  co> 


,      9  =  cos 


'^=  cosh 


-1   1 


and  hence  (3.l6)  becomes 


'.a 


I  Ifk  -/k-il  I  </  jwifj    Jhdr^  ^ 


cosh(2k-2)^y   ""f  cosh(2k+l)w  ^   cosh(2k-lV) 


A  111^^^ 


(3.17) 


For  the  successive  overrelaxation  method  with  ^^^^  =  D'V^^^^  -  s|^^^)  and  b  ='S 


we  have 


V^>  =^ 


m 


I^IL.  .  1 )      sin(m-l)Q       cos  mQ 


9  =   cos'      -tL      .  h   - 


1  +  A  -  ^ 


=  /^&TT 


so  that 


.^      .  _\l/2 


C2[p2^1  .  (2k^i)    /TT^j  ,  ^,  ,   (2^.,)  ,^T^2)Jj  I  1^(1)  I 


(3.18) 


Let  the  proper  vector  associated  vith  u,   the  largest  proper  value  of  G, 
be  D-^/^i.     Pr„„  (3.15)  „e  see  that  if  ,<^K  ^^/^Z^^) ,  a  a  scalar,  then 


for  the  modified  Chebyshev  method  and  successive  overrelaxation  method  when 


(3.19) 
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We  wish  to  consider  the  successive  overrelaxation  method  without  the 

assumption  that  ?$^^  =  D'-^ft^^^   -  s%^^^]         t?o>.  +>,<.  ^        4.  .   .^ 

/^l  2  V  -^      %  /  •   -^^^  ^^®  °n®  parametric  iteration 

scheme,  vre  have 


•k-i  1^  >  1 


and,  hence, 

Then  since  the  difference  equation  for  ^^  -  J^_^   is  identical  as  the  difference 
equation  for  ^^  -  1  given  by  (2.29),  the  development  of  the  bound  for  |  |J^  ^   f      \\ 
is  identical  to  the  development  of  the  bound  for  I  |/j^  -  ^|  |  .   Hence  as  in  section 
(3.6),  we  have 


Mfl^  ^iC^ffl  •  4.A|l§i' •  <*  -  .>Y4. 


2k'^y  |D 

where 


m 


IV   THE  MBERICAL  "BEmVItK  t^  "THE  ^EHERATED  SEQUENCES 

4ol  ^e  numarlcal  process 

m  the  previous  chapters,  ve   have  discussed  the  behavior  of  the 
se^ence  of  approxin^tions  to  the  solution  of  a  system  of  linear  equations 
under  assumptions  that  the  calculations  ^re  performed  exactly  and  that  there 
vas  no  error  in. the  given  data.  In  any  calculation,  however,  the  numerical 
operations  are  .replaced  by  pseudo-operationsc  Furthermore,  it  may  be 
neceasaiy  to  replace  A,  5^,  G  and  H  by  their  digital  approximations.  In  this 
chapter,  we  shall  consider  the  effect  of  the  above  inaccuracies. 

The  basic  iteration  process  described  previously  is 

^+1  =  2c^+l^^^  +  ^y  -  ^k_i)  ^-/k-l    (k  >  1)      (^.l.a) 


with 


nd^  givene  We  wish  to  consider  the  case  when  t^_^^  =  b^_^_^,  the  coefficients 
associated  with  the  Chebyshev  iteration  scheme,  and  c^  =  b  for  all  k. 

In  order  to  proceed  in  our  analysis,  we  shall  make  some  assumptions 
about  the  coo^utationo 

lo  The  computation  is  c^fried  out  on  a  digital  computer  in  fixed 
point  c 

2.  All  digital  numbers,  S,  lie  in  the  range  -  1  <  x  <  1  and  are 
represented  by  a  ^-  place,  P -  base  digital  aggregate. 


3^  H,  the  digital  T^^pr.^Maxr.on  of   H,  is  positlVB  definite  and 


symnetric  <. 


^.  The  proper  va.i.ues  of  G,  the  digital  representation  of  S,  are 
let.b  than  one  in  magnitude. 

E-^en  though  all  the  elements  of  p^..may   be  less  than  one  in 
HHgnitude,  there  is  no  assurance  tnat  all  the  elements  of  the  generated 
vectors  uill  remin  less  t^an  one  in  magnitude.   Frequently,  vhen  a  problem 
is  solY^.  -it  is  possible  to  scale  it  .before  it  is  placed  on  the  computer. 
We  sb^l  aot  assume  that  this  b^s  been  done  here.   One  can  either  scale  each 
toment'of  the  generated  vector  individually  by  some  power  of  p  or  scale  the 
entire  vector  by  a  power  of  p .   If  the  elements  are  scaled  individually,  then 
for  the  nmnerical  process  {k.l),      it  is  necessary  to  store  2N  exponents. 
They  may  be  stored  separately  or  stored  with  the  numbers  themselves.  If  they 
are  stored  with  the  numbers,  then  some  of  the  digits  of  the  numbers  are  lost. 
If  they  are  stored  separately,  then  the  storage  required  is  increased.  We 
•shall  assume,  then,  that  all  the  elements  of  the  vector  are  scaled  by  some 
power  of  ^c 

The  initial  vector,  ^^^  is  scaled  by  (say)  p^  so  that  within  the 

machine  ve  have  f^^^   f  /^  (i  =  i,  2,  ,..,  .w)  ^ere  f  3^^  indicates  a  right 

shi-ft,^f  p^  pilaces.  If  during  the  calculation,  a  number  exceeds  the  capacity 

■:of  a  xe^stfir  or  is  about  to  do  so  (such  as  for  a  division),  then  the 

exponent  ^p  lisjcreasedo  Thus  there  ±s   asstsciated  with  each  machine  generated 

_       p., 
•vector,  ^^^  1  T  P  ""  (i  =1,  2,  ..o,  n)^  a  power  of  p,  p^,  and,  hence, 

%  -  %  ^  •"  °  -  Pjc'  ^^  ^^^   ^j>   ■theaa  either  the  computation  should  be 
stopped  since  tbe  range  of  the  answers  ^exceeds  the  capacity  of  one  register^ 
or  the  iterates  must  be  stor^  in  more  than  one  register o  Since  the  p  '- 


form  an  Increasing  sequence  and  are  boimded  by  c,    we  shall  assiane  that  for 
k  >  K 
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%  =  P  K+l 


The  actual  computed  vector  will  be  denoted 


P, 


as  ^,  —  3  ,  and  thus 


(4.1)  becomes  during  the  computation 


?k^ 


?] 


k+1 


.  „^k+l    - 

■^  ^     =  2=k.l"l 


ks  .  .^k+l-Pk-.    ,^   ^.  .  .Pk+1 


Hf^^'k  T  P  ^)  f  P  "^"  "]  +  (H^^y)  r  P 


(?    ^  B^k-1.  ^  Pk+1-Pk-]J 
7k-l  •  P    ^  "  P 


+  (f    ^  A-l^  •  A+l'^k-l 


(^k-l^^^'^)TP 
(k>l) 


With 


0^  .  A-Pq 


7i  7  P       =    G^^f^^G  ^  P    )  f  P         ]  +  (H^3y)  f  P 


(if. 2. a) 


(if.2.b) 


where  ic^^,  ^,  ^  represent  pseudo  operations  which  may  be  dependent  on  G  and  H 

.   A+l"Pk-i   ,  .        ,      . 

U  =  0,1;   indicates  a  right  shift  of  p  -  p      .    (i  =  0,  l) 

places.     Eqmtions  (4.2)  may  be  rewritten  as 


and    f  p 


k+1 


_.  .z? 


^k+l-Pk- 


A+1  7  P  ^^'^  =  2c^^^  JG[(^^  f  P  -)  ^  P--  -]   ,   (Hy-)  f  p^k+1 

Pv    1  Pi,,n-P 


(^k.i  r  P       )  -  3 


k-ls    .      -^k+l'-^k-l   7        ^  P 


Pn.  ,  n  -Pi 


k-lv     -    ^^k+1  ^k-1 


^l. 


(k>l)  (4. 3. a) 

.:?     .     Pi       _    J*  Pn  Pi-Pn  r»  Pt        ::> 

7i  7  ^      -  <5E(^^  7  P     )  T  P  ^    °]  +  (H?)  r  p  1  +  /q      (4.3.b) 


where 
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1  =  2c, 


^\.i-iY-2^^k  r  ^'')  ^  ^'""'■'"]  .  (H.3?)  r  ^''"' 


_  ^^         ^  A  ---^  ^  «^k+l"Pk-l 


k-1- 


(7k-i-p     )"p 


-2Vi[^f(;k  ^  ^'')  r  P'^^^"'^].  (H^)-  p'^-^  -  i$^_^  , 


-  ^^^-^) 


Pk+i-Pk-ii 


(k>l) 


and 


We  shall  again  rewrite  (4.3)  as 


p'^  -  g4  ,  p^  .  g'l-^   .  (h|,  .  3^1. 


fk.l  =  2=k+l[%  *  ^  -  ?k-l  j   *  /k-l  ^  A 


?i  =  7o  +  =y  +  ?o 


(4.1t.b) 


where 


r^.. 


k+1 


/ 


-P 


k+1       ? 


(fk.iP  '^^  -? 


k+1 


i  A*^) 


±      Pjj,     p,,_, ,  -p, 


^w.^WK  ^  p ")  -  p"''^' ''  -    f,  p""'^^'] 


+  m)T^^*^  -tfi"^*^ 


*  [(1  ,  -  p'-^-i)  ^  e'^^-i'^'^-i  -  }^_^  p-'^'^-i] 


f.-i  r  ^''^-^)  -  ^ 


^  C-x  r  3 '^-)  ^ /-^"- -  f,.,    e"'^-- 


(k>l) 


and 


0       P 


«^  +   f^- 


0  ■   (fi3        -?i 


Pi  -Pn         ^  -Pn 


^^^).5[(?o^^^")^^"^  °-/o  ^^'] 


,-5,      .  Pi         ..:*         -Pn 

+   (Hy)  f     P   "-   -  By     ^     ^ 


I/e  vn.sli  to  determine 


Now  by  (A.l),    (A.3.S.),    (A.ll.a) 


an  ^pper  iound  for  j  ]/^|  |    (i  >  i)  ^„^  j  |^  1 1 


k+1, 


f  9  ''"■■'1 


-^l=k.i#IJlf,TP^fg'^-l-"'^.^^ 


k+li 


^k+1       .-     -Pk+i, 


+    II(By)  r    3  "-^^  -  Hy    p^+l| 
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Pv^T  -Pn 


^ii(?.-i^3''-^)-  ^"^-ii  p'^^^^ii 


-  ft^'^-l^   •  =Vl"^k-l     i  -P 


^11(^-1  7  P'^-^)-g 


k+1 1 


where   |  |/  |  |   <   |  |/|  | 


(k  >1) 


Now  if 


W,ir& 


k+1 


,^k,lp"'^"'l<f^3'''^"^ 


"k+1      r?  -P 


I  l?k.i  ^  P  ''    -  fn+i   P    '''I  I  <  /i  i  :^  p'>^-i 


Aiso^ 


Pv^i -P 


I(fk,irP')r3''^^''=-;,^^   p"'''-i| 


<f  ^p'^k+i-Pk       -K+i-Vi' 


^fe 


Hence  for   |c7|   <  1, 

p 
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I  K\  I   <3    ^'-^  h  l/l  I    .  /ief  p'^-1  .  2[iG|^/N   (^6  -  3~Vl-P.  ^  ^-(Pk.l-P.)  ^  ^  ^P,j 


/.  .  Pv^i  -P. 


/^6  7  P  ^'^  +  /i"  (f  ^  p^k+i-Pk-i  ^ 


-(p 


k+1 


-\J ,  ^  \.i 


i  - 


./f  (^  -  p^k.i-Pk-i  ^  ;(Px.i-p,.i)  ^  ^  ^p^,i-i>k-i^ 


and  similarly 


llfoll^/ff^-  /^noi^yf^f  3"^"'° .  p-^^i-V,  . /^1 


+  /N  ^73^ 


where 


We  define  x  to  satisfy  the  equation 

(I  -  G)  X  =  Ey 

S.=h  a.  ?  exlats  since  all  the  proper  values  of  G  are  less  tl,an  one  In  magnitude 
by 'assumption  k.     Hence, 


Gx  +  Hy  -  X  =  0 


-^-^ 


+  X  . 


and  .thus 

±        _ 

^  =  2c^^^  |Gx  +  Hi^  -  X 

Subtracting  (1..4.a)  from  (1^.6)  and  (4.4.b)  from  (4.5),  we  have 

^.1  =  ^\.i^K  -  \-i)  -  \_i  -  ?. 


(^.5) 


ih.6) 


where 


1  =  Gd^  -  ;^- 


(k>l)   (1^.7, a) 
(^•T.b) 


4.2  The  numerical  behav^^ 

We  now  proceed  to  develop  an  upper  bound  for  |  |  J  p'^^  -  ?  f  /^|  | 
''^^''  Vl  =  ^'      ^^  \+i  =   ^  for  all  k,  then  (4,7)  becomes 

^k+1  =  2bG\  ^  (1  -  2b)  \_^  -\  (k  >  1)         (4.8.a) 


^  =  ^^  -  ^0  •  (4.8.b) 

The  solution  of  (4,8)  is  obtained  in  the  same  fashion  as  that  of  (2.30);  only 
now  ve  have  an  inHomogeneous  difference  equation.  Also  as  in  equation  (2. 11), 


we  have 


G  =  FO  AC  *?"■'■ 


where  since  H  is  positive  definite  and  symmetric  by  assumption  3,  there  exists 
a  matrix  F  such  that 


FF  =  H 


(4.10) 


A.  is  the  diagonal  matrix  of  proper  values  of  G, 
0  is  an  orthogonal  transformation. 


Q  =  FO 
Then  the  solution  to  (4.8)  is 


(4.11) 


^'\  "Y^k-^  -  (2^  -  'K-l]  ^~\  -   '^'  \.,  ^''   ?,       (4.12) 


where 


j=o  ^-J    J 


=  -^:'  if  ?^ .  =  02J      --^  i  =  J 

=  0  ±^  j 
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-a  ^^j  and  ^^ .  are  the  roots  of  the  polynomial  ^ 

J     J^J  ^^V  ^>'  ~  0   •   (j  =  i,2,...if)   (4.13) 

7ZT.X'"^''  -'-''^'^■'^'  '-''  ^-),  u.),  u.a),  u.,, 

(A. 10),  tA.ll)  that 


\l/2 


(4.14) 
Wow  for  G, 

^  =  — i__ 

1  +  A  -  \'2  (4.15) 

"here^V  Is  the  largest  proper  value  of  0.  „e  shall  first  consider  the  case 
w.en  .  <  h  <  1.  P..  S  ,,  ,,,,  ,,^^  ^^^  ^^^  ^^^^^  ^^  ^^_^^^  ^^^  ^^^^^^^  ^^ 

thus 

h\i  - 


_v  1    sin  k9. 

for 

sin  9. 
J 

9.    /O 

kp^-1 

for 

9.   =  0 

Where  ^   =  /2b  -  l  and  cos  9.   =  bX./p  .      si^ce      L"^"!  SUlM  |   <  ^.k-l 


sin  9 


by  le^a  (A.2),  and  recalling  from  theorem  (2.3) 
(4. 14)  becomes 


(4,16) 


::n  general,    olio  only  i:nuv/.odga  vg..^11  .h^^e  oi'.  ^   ia  taat    WY-W   <\\f\\, 
J  >  1,   and  I l^^l I   <   I  I  ^'  I  I .     Hence  making  this  adjustment  and  summing,   we  have 
|H|     V/2  , 


H 


V 


f(^^^    M2oll-if^-Mi?'ll 


C 


1.  - 


-k" 
rN2 


k^ 


l?l 


If  1/2  <  b  <  b,  then  by  a  similar  analysis  to  the  above 

/;7k+l       sk+1 ' 


(i^.17) 


where 


2b 


k-l       3k-l 


-  (^i^g) 


2  ^r^  -  ^1 


^1-^2 


^^        1-?^ 


l?ol 


^ 


1-^1  1      -      ^. 


1^1 


(4.18). 


?^  =  b\'  +y  b^'2  _  2b  + 1 


?2      =     "b^^'    -yb'^X'^   -  2b  +  1 


Since 


1/2  <  b  <  b     and  for     0  <  \'<  1  , 
0  <  ^2  <  ^1     =     ^^'   +  /(I   -b\')^  +  2b(\'.-  1)     <  bX'   +   (1   -  b\')   =  1   . 


Then  since 
Ix    p    ^ 


-^  'o-*^  11/-  -\  K-  -^  ^\r  ■*  -1^1 


■>  -P 


ii'p^-^.-p'^ii  <  ii^iu^-^ni;,  p 


''-/k^/'^ll    •   ('*.19) 


Nov  if 


Oi£ 


'A.i  ^'''-A,i"p''l  <  e-,^^ 


'; 


(4.20) 


Hence,  we  have  from  (4.17),  (4.l8),  (4,19),  (4,20) 
Theorem  (4.1).   For  the  one  parameter  iteration  scheme  if 


-         /  2     =     "^  <  ^  <  1     , 

1  +  Vl   -  X'^  ' 


then 


H 


\l/2 


AT3 


u 


H 


!/ 


r'  p  -  >^ 


rfflN^o. 


+  V''"''  IIP  1 1 


where 


(^  =  y2b   -  1  ,      and  if     I/2  <  b  <  "5 


(1*.21) 


r'   -   ^' 


^k-1     -,k-l 


1^2' 


^1-^. 


It 


^X-^2.,. 


^1   -  ^2 


I? 


^1-^. 


1  -  r    1  -  ^: 


1-^1     1  -  ^, 


+  /Ne  f  3  "  . 


(4.22) 


minimum  of  the  right  hand  side 


We  would  like  to  get  an  estimate  of  the 
Of  equations  (k.2,)   and  (4.22)  with  resj^ct  to  k.  This,  of  course,  would  involve 
a  knowledge  of  |  [d^l  |.  „e  eaa  get  an  upper  bound  for  the  minimum  of  (4.2i)  and 
(4.22)  by  simply  letting  k  -»  a,.  Thus  for 


b  <  b  <  1 
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Wj  J       2(b  -  /iTTT)  .  p      M 

(4.23) 
Since  ^  =  /2b  -  1,  and  for  1/2  <  b  <  S 

Min  I  ||  p  ^k  _  1^  ^  ^Pk|  I  ^  ^^11^  -Pk  _  I  ^  ^Pk 

|H|  \V2      . 

fH]7       -  '   '-  ,  +yN<^-^3P  =  M'   (4.24) 
'  UJ  2b(l  -  \') 

since  (1  -  ^^)(i  -  ^^)  =  2b(l  -  \'). 

The  minimum  of  M  is  achieved  when  b  =  b.  For  b  =  "S 


H 


a/2 


M  = 


'u 


111 


|H|y  /     2(i-(^) 


+  /N  <f  T  P 


On  the  other  hand,  M'  grows  smaller  as  b  approaches  b,  and  the  limiting  value  of 
M'  as  b  approaches  b  is 


M'  ='    ^ 


iHi   W^   ,,:, 


'^U; 


^^    -/n^tP^ 


^  -Pv  ^  .  Pi 


Thus  we  see  the  upper  bound  for  the  Min  |  |x  p  ^- f^  r  ^  ^\\      is  miniinized  with 

respect  to  b  when  b  =  b  =  ^ 

1  +  /l  -  \'^ 

4.3  Terminating  the  one  parameter  iteration  scheme 

As  was  pointed  out  in  section  (2.4),  for  the  averaging  processes  one 

ca.  estimate  |  |3  -  ^J  |  from  7^  =   G^  .  h/  -  ^^  or  ?^  =  1  -  A^^.  Furthex^ore, 


for  ^^  near  the  solution  5,  both  |  |J^|1  and  |  [JJ  |  he=o.e  s^all.  „e  conclude 
frOM  (4.1)  that  t^   is  not  difficult  to  calculate  during  the  iteration  process 
vhereas  ?^  may  require  much  additional  calculation.  For  these  reasons,  ve 
propose  to  discuss  computations  Walch  use  equations  (l».£.a)  and  te^inate  when 


where 


\\\\  <  ^ 


6   Is  a.  preasslgned  number 
■^k  =  H%  ^  ^^''^  *   (5-3*)  ^  p'''=  -  f^  -  /"^       (4.25) 


and 


u>^   and  i<j^   are  defined  as  for  (4.2). 


i^ 


We  may  revrite  z,  as 
k 


k 


z,  =  rA>,     -  a^\  ^   /nit^  •  ."^k   :>  .  -   ^ 


where 

Because  of  roundoff  errors,  6    cannot  be  any  preassigned  number.  We  now  proceed 

to  develop  an  upper  bound  for  the  Mln  \\z    11. 

k     ^ 

We  have  from  (4.26)  and  (4,5)  that 


=  (I  -  s)(i  p"'''^  -  -^"^  ^  ^% .  [(^)  -  p''^  -  g;  iS  +  /. 


Now  let 

=  5(1  -AW^  [(I  p"'''^  - 1  i\  ,  (?  b"^''  .  i  .  «'i 


■A^  ^'-(A^     -A-p'^)] 


by  {k.S)   so  that  by  (A.  11. a) 
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I  \^'\\  I  <(!..')  I  |Q-^[J,  .  4  i^^  .  f^    ^  ,^^3    1 1        ^ 


Then  since 


\\\\l<\\iJ\*my)^^'^.(S§)i\,  iif-ii    , 


and 


|(H|)i  t/^  -   (h|).    p"^k|      ^^^ 
we  have  from  (4,17)  and  (4.18)  for     S  <  b  <  1 


\ 
p  ^     ^ 


!?JI< 


IH|      \V2 

^1  (l  +  \'))p^ 


H 


'ni  +  ki^)iiiji  +  kpk-i  11^, 


1  + 


C 


-2       M-Ql 


__(1  -  ^)2    1 .  ^ 


and  for     I/2  <  b  <  b 


(^.27) 


^kM   < 


HI     f/2 

"'         (1+X')M 
2b 


H 


'X 


'^f '  -  ^f  ^ 


h-^ 


5k-l       ^k-1 


+  Jn  i-f-  ^ 


h-h 


l'^2' 


»k 


!ilii^ 
^1-^. 


It 


1-?^     x-t 


1-^1    1  -  ^, 


(4.28) 


where 


l/iM<   11/' 


We  wish  to  determine  an  upper  boimd  for  Min 
letting  k-^-  in  (4.27)  and  (4.26).  Hence, 


z-|^  I  I .  We  may  do  so  by 
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Sieor^  (4,2)?   For  the  one  parameter  Iteration  scheme,  for  ^  <  b  <  1 

then 


3    /iHi  y/^ 

Min   z    < 


k 


u 

It! 


V 


^"""'^(ii^^^^^^'J^/^^^p'^^M 


U 

(i^.29) 
and  if  1/2  <  b  <  b 

(i^.30) 

Thus  if  we  choose  an  ^  which  is  less  than  the  bounds  given  by  (4.29)  or  (4.30), 
there  is  no  assurance  that  |  |f^i |  will  be  less  than  ^    and  hence  no  assurance 
that  the  computation  will  terminate. 

4.4  The:  numerical  behavior  of  the  Chebyshev  iteration  scheme 

Theorem  (3.3)  showed  that  the  b^'s  converged  monotonically  to  b.  We 
now  consider  the  behavior  of  the  b^'s.  If  we  use  the  Chebyshev  iteration  scheme, 
then  the  b^'s  are  defined  by  the  recursive  relationship 

^V+1   ~       o —  3        b  =  1   . 
2  -  X'\  1 

k 

Let  us  assume  that  1/2  is  exactly  representable  within  the  computer;  this  is 

always  true  if  p  is  an  even  integer.  Then  within  the  computer,  the  sequence 

\+l   =  -  1/27  (-1  +  [(^•)^t2]v  b^) 

\   =  -  1/2  7-  (-1+  [(\')^r2]) 
is  generated  where  ;r  and  7  indicate  pseudo-multiplication  and  pseudo- division, 
respectively.  To  analyze  the  sequence  Jb^^j  ,  we  make  the  following  assumptions: 
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(i)  If     b  <  c,    then     axb<axc     if     a>0; 

(ii)   if  b  <  5  <  0,  then  a  f  b  <  a  f  c  if  a  <  0; 

(lii)  if  a  >  0,  b  >  0,  then  a  /  b  <  Min  (a,b)j 

(iv)   if   |a|  <  |b|,  then   |af  b|  <  1; 

(v)    if  a  <  0,  b  <  0,  then  a  7  b  >  0. 
Lema  (k.3):      If  0  <  (x.)^  ^  2  <  I/2,  then  under  assumptions  {i),  (ii),  (ni), 
(iv),  and  (v),  then  the  sequence  fb^j  forms  a  monotonically  decreasing  sequence. 
Furthemore,  b  >  0,  k>l  so  that  for  k>M,  "5=5    -     -h- 
Proof;  We  have  be  definition  that 

\  =   -1/2  7  (-1+  [(>:')^7  2])  . 

Since   0  <  (\')^  -  2  <  I/2   -l  <  i  4.  Cf  ^^2  •  o  ^   w^ 

_  \      J      .  ^  ^  i/.f,   1  <  .1  +  (X  )  7  2  <  -  1/2,  and  hence  by  assumptions 

(iv)  and  (v),      0<b2<l.     Since     1  >  b^  >  0, 

[(:^')^7  2]  X   b^  <  (?:')^7  2  <  1/2 
by  (iii).  Then 

[(X')^7  2]  X  bg  <  \'^-2<l/2 

-1  +  [{X^f   7  2]  /  b^  <  -1  +  (x^f   ^  2  <  -1/2 

.l/2^(.l^  (X.)2-^2)    >  -1/27(^1^  [(X 0^7  2]  X  £^)>0 
and,  hence 

^2  -  ^3  -  °  ° 
Now  assume  1  =  "b^  >  b^  >  b^  >  , . .  >  b^^.  Since 
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JX 

by  assumptions  (.1)  and  (iv),  and  hence 

0  >  -1  ^  [(X'f  f  s]  ,  i^_^  >  .1  ,  [(^.)2  ^  2]  ,  ^-  _ 
Thus  by  assumptions  (ii)  and  (v) 

-1/2  7  (-1  +  [(X-)'  f  2]/  Vl'  ^  -V2  7  (-1  +  [(>:')'  7  2]  V  bj  >  0 


or 


Thus  we  see  that  the  h^^'s  degenerate  into  a  constant/ b',  f or  k  >  M 
and  then  ve  have  the  one  parameter  iteration  scheme.  When  b^^  =  b',  then  we  may 
use  either  process  1  or  process  2  which  are  described  in  section  (2,9).  In  view 
of  theorem  (2,6)  which  is  an  exact  analytic  result,  it  would  seem  that  process  1 
is  more  appropriate  than  process  2,  Thus  once  the  b^'s  become  constant,  they 
shotild  be  allowed  to  remain  so. 

The  bounds  given  by  (1^.23)  and  {k.2k)   for  Mln  I  I J  s'^^  -  t    ^  r\  I 

i  -P    i     P        k  /k  •  P   I  I 

were  arrived  at  from  lim  ||x6  ^  -  ■>;     -ft  ^11    q-ir,^^  +v,^..  -k 

xm  I  I A  p     /  V  ^  P   I  I  •  bmce  these  bounds  are  inde- 

pendent  of  the  initial  error,  and  since  the  Chebyshev  iteration  scheme 
degenerates  into  the  one  parameter  iteration  scheme,  these  bounds  are  equally 
valid  for  the  Chebyshev  iteration  scheme.  Similarly,  the  bounds  given  by  (4.29) 
and  (I..30)  for  Min  |  |lj  |  are  independent  of  \\%\\    and,  hence,  are  applicable 
to  the  Chebyshev  iteration  scheme.  Thus  the  termination  procedures  for  the  one 
parameter  iteration  scheme  and  the  Chebyshev  iteration  scheme  are  the  same. 

h,3     Nmerical  behavior  of  the  ge^^ 

In  chapter  III,  it  was  shown  that  if  a  matrix  A  has  property  (A)  and 
ordering  ^^,   then  for  the  operator  G  =  I  -  d'^  it  is  only  necessary  to  compute 
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(2)   fo^  the  iteration  scheme  described  by  {k,l).     We  shall  assume  in 
/2k+l  ' 

this  section  that  if  a  matrix  A  has  property  (A),  then  A,  the  digital  representa- 
tion  of  A,  also  has  property  (A).  Thus  for  the  numerical  process  described  by 
(^.2),  after  k  iterations  ve  have 


^k 


>k73 


Note  that  we  have  modified  the  process  slightly  so  that  the  vector^  has  only 
one  scaling  factor  associated  with  it.  Let 


"^  "    '   J(2)   ^(2) 


^2k+l 


2k 


/     \   2k+l  / 
we  now  proceed  to  describe  an  upper  bound  for  | |?  | |  when  c  =  b. 

Since  A  has  property  (A),  the  proper  values  and  proper  vectors  of 
G  =  I  -  D  A  have  the  same  properties  as  described  in  section  (3.2).  We  denote 
the  proper  vectors  of  G  as  D-^/^f^^  T^^%   ^'^^%,   D'^/^f.   so  that 

where  M  is  the  diagonal  matrix  of  proper  values  of  G. 
From  (4.12),  we  have 

Now  let 

N^  -r        N  -r 


t  =  6-^2  Z 


^       t  ^  ->  1      _>    '2 


'°  '  "    /itl^^'i  ^  i?,^l-'i  '   ,?x^^2^1  ^  ,^,^^M,.r%        (^-32) 
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and 


-♦ 

5 


=  D' 


f:^.l/2  J    V 


N  -r 


Mg-r 


L  "^'^'^ '  ifi"/.i-^i  ^  ,5,  "^.i.sA  -  ^f^  -/,i.H  .A 


Then  (4.3l)  becomes 


t    = 


-1/2  r""         -»     r  ^    \-^  N  -r 


/-O 


N^-r 


J,°y,i^k./^i)^  ^  1^  «/,i.A./-Si)?i  -  J^  «4i.2A-/°)Pi 


where 


N^-r 


S  (fl.) 


"^li   '^21 
^li   ^2i 


Z.   a/>  .  ^T   S   .(0)q 


if  ^ii^021 


=  ml 


;7J 
.0 


m-l 
li 


if  ^,  .  =  ^ 
^li   ^2i 


and  ^11  and  ^^^  are  the  roots  of  the  equation 


^j  ■  ^"^^J^j  +  (2b  -  1)  =  0       (j  =  1,2,... N) 


and 


V^j^  -  V^j^^j 


Ml. 


'k-l^-j 
It  is  not  difficult  to  verify  that  S   {-{1)   =   (-l)^^X(^.),  and,  hence. 


S^(0)  =  0  if  k  is  even.   It  is  also  true  then  that  \{-Jl.)  =  (-l)^  T  (jl.)  and 
hence  T^(o)  =  0  for  k  odd.  Then  in  view  of  theorem  (3.1), 


2k 


1=1 


2k-l  — 
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1 


N^-r  ^ 


1=1 


2k -1 


^.2rV0)-/^     "y,1.2Ak-/^l) 


and 


2k+l       ^2  )    ^ 

/1=1 


r    I- 


2k  - 


f(2) 

1 


N^-r 


2 
+     Z 


'2k 
Z    a 


1=1  L/=l     ^^i+V^  2k-/+l 
Then  by  theorCTi  (3.2),    we  see  that 


(0) 


1^(2) 


isy^jd) 


2    _    1       y. 

'1     "2k    "      -  2    .    , 

1=1 


2k-l 
(a.    -  a        )T     (u    )   +     Z     f 


-l/c 


l'   V=0   *"^.i  '   (-l'^'^/,l.r'S2k-/^l) 


N^-r 


2k-l 


+ 


,5,    K2r-2.(°)-^f„-y,i.2AM'^l' 


and 


Jd1/2-(2)    1,2  _1     J 


2     "2k+l 


1  r  2k 

-    Z       (a     -  a       )T         (a.)  +     Z     (a,   .   +  (-l/a^         )s        ,     (n    )T 

"^  1=1  L     1  1+^     2k+l     1         ^^Q      /^i        V      >    /,l+r^''2k-y+l^''^l'' 


N„-r 


2 
+     Z 


"2k 
Z    a 


1=1  Uo     /M^+^  2k-/+l 


(0) 


Hence, 


:dV2?m2 


l\f  -  w^'Ml^if .  \iti/%ii\f 
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i    Z 
2  1=1 


(a, 


2k-l 


1  1+r 


K^^^,^^'£(-^,,^(-^)\,js,,./^^) 


1     V 


2k 


1  i+r'   2k+l^ 


s^ 


(-.    -  a,.jT_,(^J  .^^^a^^^  ,  (.ir,^  ...^S^^_,^^C^J 


/,l+rr2k-/+l^^l' 


N  -r 

+     Z 

i=l 


2k-l 


-|2 


+     Z 
1=1 


_    2 


'"/,M,+Ak-7+i(°) 


N 

z 

1=1 


/i^/o 


I'TOI*    •••    Vsk,! 


where 


(a.    -  a.      ) 


^"l   -  Vr^ 


^2k+l('^i) 


1  =  1,2,. .,r 


1  =  r+l,    , . .  _,    2r 


=  a.    ^  T     (o) 
i+2r  2k 


i  =  2r+l,    ..._,    N  +r 


and 


=      0 


1  =  N^+r+1,    .,.,    N 


f,.  = 


(a 


^ 


>,i  ^  <-^>  °/.i..) 


/^ 


^2k.^(^l) 


=  1,2,  ...r;   A=  0,1,2,. 


..2k 


/ 


(a/  .  +  (-1)  a.  .     ) 


/2- 


2k 


->1^^1^  ^  =  ""^^^    '">    2r 


=  ^^,1.2Ak-^(0) 


1  =  2r+l, ...,N^+r 


-"/,l+N^+Ak-/fl^°^ 


1  =  Nj^+r,  ..._,N 


Then  we  see  by  (A.l)  that 


^3 


^'^'^ii^/i(;i^ro,.--/..,,r 


N  2k        pN       ~ 


If'-llJjk^-lfJjU. 


/jA".-'../(4>-.)-4.A)-"i 


Z       _  . 

^-|_       l+2r  2k 


^.oT1(0) 


yI/ '  A(v,i  ^  V.g^^4.e.(^.)^  3^2^.2,..(^,j  ^  |^f«^,,...^.AWi(o) 


N^-r 


>l/  ^  lfi*"2/-l,i   -  "2/-l,l.r)'(4-2/.l(Si)-  CaA^^l)    -    2^  "sM,l.Sr4-£r.l(0) 


r,2     /      N  „2 


^     ^n^,/^2K(^)   -  W(-)  y?       2     (a,    -  a.^J^  ,     Z^   -^ 


CK^O^.V      i^K-  2k+r-'  ^  2      .-^   V-    -  a^_^^;      +   ^^^    a,_^g^ 


^  <k7<..  /?>'  ^  4.A)  /Tf^^^w^TT^uir 


*  ii  o5<,/'2'>^-2^'^>  ^  4-.m(^)/ i  J^Ca^,.,  ^  «2/, 


N  -r 
n2  y       2 


N^-r 


^  A  <^  '^^-^^^^'^  *  4-a/..(^)  /|  ifx'"2^-l,i  -  '^2M,i.r)'^J^  «I/.i, 


i+2r 


From  (^.32)  we  see  that 


9h 


i=i  -   --    i^i 


c 
i+2r 


Ng-r 


iisy^f;ii^=s  («    ..     ^^.  z- 


N. -r 

i+2r 


ll^i^'^a^-ill' =  i '"a.-x,.  -..-M./ ^  Y'"^.,. 


SO  that 


Now  for  G  =  I  -  D""*"!  , 


where  ^'  is  the  largest  proper  value  of  G.  We  shall  first  consider  the  case  when 

/J 

b  <  b  <  1.  Then 

,  ,   sin  kO , 
V^^i^  =r    ^      for  9,  i  0 

"^  sin  9.  J 

,  -k-1 
=  V  for  9.  =  0 


where        p     =  /21b  -  1 
and 


and         cos  ©     =  h^x./p     , 


By  leimna   (A.2)   and  theorem  (2.3),    (1+.33)  hecomes 

liiii<r'' 


1  +  2k 


r2  \2 


-2 


(5^(1  +    (2k  +  1)   ^^^l£ 


-P^Y  /|Dj 


1  +f' 


D, 


/lT^)(2k  +  1)  ^^^-^1    '^2'u 


1/2 


D 


1%^' 
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+     Z     (2k 
^=1 


2/  +   1)  f 


p:2k-2/-l 


D   I     \V2 


D 


+     I     (2k   -  2/+  2)  <? 


A-± 


In  general,    our  only  knowledge  of  f^^\    ff^\    }i^) ^      ^iH  ^e  that 


0  '    ^2/  '    ^2/-l 


irf'll<ll?,'^^[|,  lll<5^ll<l|T(^)l|and||a)<||T(l) 


2/ 
this  upper  bound  and  sunnning,  we  have 


;:2  \  2 


V        1  +  r/         V  i  +  f 


Then  inserting 


Jul  ||J(1) 


D 


u 


U 


+   2 


r2>k+l 


:2,k 


1   -   (f'")""        (k  -^  1)(,°-^) 

1  -  e^ 


(1   -  ff 


%\y 


fi„i  Y/2„-*( 


51;  /  I5L  2 


X.f^ 


where   /o  =  \/  2b  -  l 

If,  however,  1/8  <  g  <  ,",  ^hen  ^y  .   slMlar  analysis  to  the  above 


"^11  s  liJ^h^zi^ r-^ 


1/2 


where 


2k>^'-   -2k+l' 


W'. 


2b 


¥2  I    ^1  -  ?, 


2k+l       ;,2k+l' 


ltd) 


D    I      \V2 
2'u 


Id 


U 


ll?^^^li 


A_  Pi  A  -  ^1 


2k 


1  -^: 


2k 


^i-^2L/^2i-^fr/^  rr^i  j^    ii'^^^^ii 


r/'^ 


^Y^' 


uy 


1  1     A  -  (^1) 


^^k+l^ 


^1   -  ^2  K^1^2    U   -  ^1 


1    A  -  (0p) 


2\k+n 
2 


hh    I    1   -  ^: 
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D    I     \V2        ^ 


^U 


^1     =  bil'    +yb^^'2   .  2-b  +" 


(^.35) 


^o      =  "bi^'    -/^u'^   - 


^t'      -  2b  +  1 


and 


for     1/2  <  b  <  S  , 


7in-l       ;7in-r 


m 


^M     ^1   -  ^2  ^  ^  ^,    -  ^. 


1        '"2 


Then  since 


|x  3 


k       ^      '  ^  ki 


-Pi 


we  have 

Theorem  (h.k):     For  the 


X 


-^k   -^ 


successive  overrelaxation  scheme,  if  b  <  g  <  1  then 


P-      -     -P.  p 


^  "  ■ /k  "^  ^^1  I  <  i  |d  I  I  ft  ^'k  -.  nr^  ^  o^k 
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where  an  upper  bound  for  |  |d^| ]  is  given  by  (4.3^),  and  for  l/2  <  b  <  S  then 

vhere  an  upper  bound  for  |  |'^|  |  is  given  by  (4.35). 

mie  method  of  termination  for  the  successive  overrelaxatlon  method  will 
be  to  stop  the  iteration  process  when 

where  ^  is  a  preassigned  number  and  k  >  K.  If  <f  is  too  small,  then  there  is  no 
assurance  that  the  successive  overrelaxatlon  method  will  be  teiminated.  We  shall 

now  proceed  to  develop  an  upper  bound  for  Min  II?  -  B^  -  ?    ^  ftP| I 

k>K   ^^  '     •>^'^   •   "  • 

We  note  that  for  k  >  K 

-  fill  ^  ^^  -  ^'^'  3-  -(fil  ^  P^  - 1<^' .-)    . 

From  (1*.3),  we  may  develop  the  equation  for  x  p-^  -?   ^  pP  m   exactly  the 
fashion  as  the  equation  for  t  -  V^  was  derived  from  (4.4)  and,  hence, 

-  (ab  - 1)  rV2o  s^_^  0*  ei/2(i  e-p  .f^^^p 


same 


q-1 


i     D-V2  0  S      ,   0*  W^  /     . 


where 


/^^  is  defined  as  for  (I1.3).  Then  expanding  ?s"-P  -  ^   -  a? 

X  3  P  -  ^j^  -^  pP,  and  /^_^ .  in  the  proper  vectors  of  G  and  proceeding  exactly 
in  the  same  fashion  as  for  (4.33),  we  have 

+     E         Max      |S^     ^>^(n)    -S  ,rLi)l      llfiV2/(2)      ,, 

q-2 


S"3 


Let  us  consider  the  case  vhen  b  <  b  <  1.  I„  [8],  It  was  shovn  by  Young 
that  if  b  is  Slightly  larger  than  b",  then  the  Increase  In  the  number  of  Iterations 
necessary  to  reduce  the  Initial  error  by  a  fixed  amount  Is  relatively  small 
whereas  If  b  Is  less  than  b'  then  there  Is  a  Much  larger  Increase  In  the  number 
of  Iterations.  For  b  In  this  range  all  the  roots  are  complex,  and  thus 

Bin  an9 


\(^j)=r" r^     for  5=0 

"^  Sin  9.  <3 


=  ^P  for  9.  =  0 

^  0 


where   ^  =   (2b   -  l)^/^.      Then 

Max      |S   (^x)    -  S        (n)|    =     Max        l^"^'-  sijLm©   _   -m+1  sin(m  +  2)9    , 
0<n<n'     ^  °i+2  o<n<7r/2  ^  sin  e       ^  '       iiT© " 
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=     Max  \f^  r(p"l  _  (5)   sin(m  +  l)  Q 

0<9<r^2  L^  ^  sin   Q 


sin  0 


^— 1 


cos  0  -  (j5-  +  ^)  cos  m©  1 


;rm  r/^-1 


<f   [(f"   -p)(in+  1)  +  (^-1+  p)j 

As  a  consequence  of  the  above  inequalities. 


H^"^^%.,-l^^'-U,r.'\\ 


<  Max 

"  0<M,<|I 


,  l^£,-2(^)  -  ^^Ml       I  |dV2(^(2)  p-P  .  ^^^^^  .  pP)  I  , 


'   0^^.''2*-3'^^  -  Vl^-^l  llSy^t''^^  P-^  -;2K^P^)|l-|2fi  -  H 


=-2^^    c^ 


1  -  (f') 


•fl'-T^' 


52^q-l 


i5jy^ii?^'ii 


Ct.se) 


where   |||(2)||  <  ||j(s)| 

it  can  be  shown  that 


Id  1^2    |!f(2)       _^     p        ^         . 
2j/         ljK+q-1-P      ^Jk+q'P 


^nd      Mf^)]iM   <    11?^^^  I  I    .      In  the  same  fashion. 


-  o5<,.l  Vl(^)    -  Wl(^)l    I  l^2'/^(^'^^^   3-   vIkIi  ^  3^)1  I 
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.  \2i  .  l|  ^^.Js^,Jn)  -  s^^(.)|     ||5V2(?(l)  3-P  -;(^LpP,|[ 


0<|J^)a 


/^2K 


^2xq  .^2sq-r 


;:-l 


''-'■^^^Jf^-f^    ^r-f 


)  ^^iO 


s2^>l-l 


•J'-'itW-'^f^j'H': 


1  -f 


i£!ll. 


ri^aiy^iil'^' 


^iM'J'w?'^ 


+     D, 


ly^ii/^^ 


(i^.37) 


We  wish  to  determine  an  upper  bound  for  the  minimum  of 


I  'jK+q  "^  ^      "  Jx+q-l  "^  ^^1  I      ^^^"^^  is   independent  of      |  \t^^^  p"?   _  4^2)     ^  ^p^  | 

We  do  this  hy  allowing     q  -^  oo  .     We  have  from 


and    |l|(i)^-P  -;^^^:^pP 

(^.36) 


Min 

q. 


?(1)       ^aP       ^(1) 


lljlcitxf^^-J^i^^^llS   "HC-,f^'^-/[i^.^|| 


< 


2(r"^  -  r) 

(1  -  r^)^ 


r. 


q-*  oo 


D, 


K+q 
\l/2 


D 


C 


^^U 


-f" 


1/2 


D 


k 


U 


?(1) 


.M",r> 


4 


^r„?(. 


l-f-^|BJ^ 


"^'^'"^ff^J( 


i/)Vl^2l.v^,, 


D 


l^ 


l/(^' 


and  from  (^^.37) 
\{2) 


r  ifeir.^  -/^^3^ii  <  -  iijit,^^^  -ii:^f  3=^11 
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-1^\  fhKT  ^^ji) 


\  -  p^   /  \  ID' 


r 


/ 


Then 
Min 


SI  V/2 


f"      |5i 


'''"'     lll'^'ll.s 


fcLu!  1/  '"s'u 


:i^ 


5  I  \  1/2 


-2 


^   -C      /I   ID 


.r?2) 


'X 


2'u 


1  -  r'  1 1^1/ 


lU 


(2) 


;-l 


f^)fii7r"i'"" 


ri  1  +  (^ 


Id, 


<     2(--     ^-1-^.    /    -ilii      li/^^-^l|^+     ^-^      li?^^||^ 


1  -  f  V      ID 


Thus  we  have 

Theorem  (4.5):  If  k  >  K  and 


'A 


V 


Jii 


/\ 


1  +  l/l  -  il'^ 


=  b  <  b  <  1,  then 


gll>.-X^P'^-jK..^^^il<2f 


Min 
q>l 


■1  /l  +  r 


Hence,  if  we  choose  an  t   which  is  less  than  the  boimds  given  by  theorem  (4.5), 
there  is  no  assurance  that   |  \j^^^_^   ^1  pP  .  J^_^^  ..  pP|  |   ^^  ^^  ^^^^  ,^^^  ^ 
and  thus  no  assurance  that  the  computation  will  terminate. 

As  was  shown  by  lemma  (4.3),  the  b^'s  computed  for  the  modified 
Chebyshev  iteration  scheme  will  become  a  constant,  b',  for  k  >  M.  Thus  the 
modified  Chebyshev  method  degenerates  into  the  successive  overrelaxation  method. 
When  b^  =  b',then  we  may  use  either  modified  process  1  or  modified  process  2. 
In  view  of  theorem  (3.6)  which  is  an  exact  analytic  result,  it  would  seem  more 
appropriate  to  use  modified  process  1  rather  than  modified  process  2.  Thus 
once  the  b^^'s  become  a  constant,  they  should  be  allowed  to  remain  so. 


Min 
q>l 


lim 


The  bound  given  by  theorem  (4.5)  for 
'  '/K+q-1  ~  ^      '   fx+q  t  P  i  I   ■^as  arrived  at  from 

||(1)    •  aP   S(l) 


Since  these  boimds  are  independent  of 

and  since  the  modified  Chebyshev  iteration  scheme  degererates  into  successive 
overrelaxation,  the  bound  is  equally  valid  for  the  modified  Chebyshev  method. 
Thus  the  termination  procedure  for  the  modified  Chebyshev  method  and  successive 
overrelaxation  are  the  same. 

h,6     Bounds  for  the  error  vector 

Up  to  this  point,  we  have  only  considered  the  errors  caused  by  pseudo- 
operations  instead  of  the  true  numerical  operations.  However,  there  are  other 
errors  introduced  because  G,  H  and  y  are  digitalized  quantities.  We  now  develop 
a  bound  for   |  |x  -  P  ^(^  -  p  ^)  I  I  . 

Now 


+  p  ^ 


(I  -  G)-l  [(I  -  G)(l  p"'"^  -j^  ^  ^%   ,   (hI)  -  p'^K  .  (g^*p-Pk  ^  jH 


'h 


and,  hence. 


'■'^J 


103 


Inasmuch  as 

X  -  X     =     (I  -  G)"V  -  (I  -  G)'^y 

=   [(I   -  G)-^  -   (I   -  G)-^]  Hi?  +  (I   -  G)-l  (Hi?  .  Hy) 

=   (I   -  G)-l  (G   -  G)(I   ..  G)-l  H?  +   (I   -  5)-l  (Hi?  .  Hy), 

.  .^      ♦                 |g  -  g|  , ,  ^      _•» 

|13-K||     <    — 1 Jia ||h|||+    IM^LJdl         • 

|I  -  °>  |I  -  G|/  |I  -  §1^ 

^^  '^Ij'ljl  -  S'   '°  -  '''i  ^  "S  "y  (A-6),:  and  If  l(Hy)^  -  (Hy)J  <  h, 
I  |Hf  -  Hy  1 1  <  /f  h.  Then  by  (A.  5) 

|I  -  SJ^  -  |I  -  G  +  G  -  aj^  >   |I  -  Gj^  -  |G  -  0|^   >   |I  -  g|_^  -  Ng   . 
Thus  ve  have;, 
Theorem  (4.6): 

Where  |  |/^|  |  <  |  |/J  |  |  . 

Thus  if  Pq  =  P^  =  ...  -  Pj^  =  p,  and  if  p  is  known,  it  is  possible  to  detemine 
I  I  2^1  I  SO  that  1  |x  -  p  \y^   7  3  )  I  I  <  ^  » 


IQk 


V    A  NUMERICAL  EXAMPLE 

5. 1  Introduction 

Up  to  this  point  we  have  discussed  a  number  of  methods  for  solving 
the  system  of  linear  equations 

where  A  is  positive  definite  and  symmetric.   In  this  chapter  ve  shall  discuss 
a  specific  example  in  which  the  Illiac,  the  digital  computer  of  the  University 
of  Illinois,  was  used  for  performing  the  calculation. 

The  system  of  equations  which  was  solved  is  the  following: 


1 

0 


0 


-1/2    0 
-1/2  -1/2 

0 


0 

-1/2 

-1/2 

0 

0 

0 

-1/2 

-1/2 

0 

• 

• 

I  ■ 

0 

0  ! 

1 

0 

0  ; 

0 

1 

0 

-1/2  -1/2 


-1/2 
-1/2 


0  I 

0  i 


N, 


I 


i 


I  I  ! 
I 


(5.2) 


1  I 


N 


with  N^  =  k9,   N^  =  50,  and  N  =  99-   Since  A  is  non-singular,  the  solution  to 
(5 •2;  is  X  =  0  where  0  is  the  null  vector. 

The  matrix  in  equation  (5.2)  arises  from  a  finite  difference  approx- 
imation to  the  differential  operator 


d^z(t) 
dt^ 


(5.3) 
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on  functions  z(t)  which  satisfy 

z(0)  =  a,   z(l)  =  h, 

A  difference  replacement  of  (5.3)  is 


1+1     1    1-1     d  z 


^I^P       °  ^  ''  °(<^*))'  1-1--..,N         (5.1*) 


'0  =  ^'    Vl  =  " 


and 


N'zit     =     1, 
We  may  rewrite   (5-^)   so  that 

^-5     (^.l^^-l'     =      ^    ft     -     o(m3)      1  =  1,..,,.  (5.5) 

Now  let 

^i=^2i  i=l,...N^ 

^i+N^=   ^2i-l  i  =   1,...,N2 

in  (5.5);  then  the  matrix  of  coefficients  associated  with  x  ,...,  x   is  the 
same  as  the  matrix  of  coefficients  in  (5.2). 

The  system  of  equations  (5.2)  was  chosen  for  the  following  reasons: 

I.  The  matrix  A  has  property  (A)  and  ordering  o^  and  hence  the 
methods  of  Chapter  III  are  applicable. 

II.  A  great  many  properties  of  the  matrix  A  are  known,  as  will  be 
illustrated  below. 

III.  Because  the  convergence  properties  of  the  methods  to  be  con- 
sidered are  independent  of  y  in  (5.I),  we  have  chosen  y  =  0  since  this 


lOo 
simplifies  the  conputati^na]  procedure. 

VI.     Since  the  Illlac  Is  a  binary -machine,  the  only  numerical 
operations  associated  vith  the  ^trlx  A  are  addition  ax.d.  shifting  «hlch  re- 
^  ,ulre  considerably  less  time  than  multiplication  for  fixed  point  arithmetic. 
For  the  Illiac  the  maximum  roundoff  error  in  performing  a  right 
shift  equals  the  maximum  roundoff  error  in  accumulating  the  sum  of  products 
and  then  rounding.  Thus  there  is  no  reduction  in  the  maximum  roundoff  arror 
committed  during  each  iteration  by  performing  a  right  shift,  instead  of  accumu- 
lating the  sum  Of  products  and  then  rounding;  and,  henca,  our  conclualons  about 
roundoff  will  be  yalid  for  matrices  whoso  elements  are  not  of  such  a  simple 
natyre . 

The  largest  proper  value  of  A  is  In-  cos  ^2.,  ^^^  ^^^  ^^^^^^^ 
proper  value  is  1  -  coe  ^.  The  proper  vector  5^  associated  with  the  largest 
pl'Ojjer  value  has  components 

fl,i    =   ^i^M        1=1,  2,...,  N^ 
^  f21— 1  ^ir 

yi,i+K'^  "   ^^^  %i —    i  =  1,  2,...,  N^ 

and  the  proj^er  vector  f^   associated  with  th,  smallest  proper  value  has  com- 
ponents 

2,i+N^  -  -^^"^+r~     1  -  1,...,  W^- 
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Then  for  G  =  I  -  d'""'"  A^ 


^  =  -os^ 


A 

b  = 


1  +  sin  ^ 


W+1 


and 


Equation  (5.2)  was  solved  using  the  modified  Chehyshev  method  and 

two  variants  of  the  successive  overrelaxation  method.  Variant  one  consisted 
Of  letting  ^/2)  ^  ^-1  ^^(2)  _   a)^.  ^^^  ^^^.^^^  ^^^^  ^^^^^  ^^^^  ^^  ^^^^_^^ 

relationship  between  ^^^^^  and  ^^^2)^  ^,^^  ^^^^^^^  ^^  ^^^  numerical  experi- 
ment was  to  investigate  the  following  questions. 

1.  How  are  the  theoretical  results  of  Chapter  III  showing  the 
relation  between  the  various  methods  affected  by  roundoff  error? 

2.  How  realistic  are  the  various  bounds  given  in  Chapter  IV  for 
the  numerical  process? 

3.  What  is  a  good  terminating  procedure  when  we  are  interested  in 
acquiring  as  much  accuracy  as  possible? 

5-2  The  numerical  procedure 

The  choice  of  initial  vector  will  greatly  influence  the  convergence 
properties  of  the  three  methods.   In  order  to  consider  the  influence  of  the 
choice  of  initial  vector,  each  of  the  three  methods  was  used  on  the  Illiac 
with  two  different  initial  vectors. 

A  set  of  "pseudo  random  numbers"  V'  (i  =  1, . . . ,  N)  was  generated 
in  the  following  manner: 


108 


^^3_  «   £.  -2   mod  (2   )      i  =  0,,.. 
'^         =  2365  6938  7927 

where  the  pseudo  operation  "-f  2^"  indicates  a  right  shift  of  five.  Let 

H    =  ^2i  i  =  1>...,  1^1 

The  first  initial  vector  for  the  modified  Chebyshev  method  and  the  successive 
^verrelaxation  method,  variant  one,  was  ^^^^  (/|  tf'^^jj      =  .119^  53^2  82) 
and  the  first  initial  vector  for  the  successive  overrelaxation  method,  variant 
tvo,vas  7(||^(1)||=   .1783  5176  4i4).  The  first  initial  vector  was  generated 
in  this  fashion  with  the  hope  that  ?q(=  |  -  ^)   would  he  a  linear  combination 
of  all  the  proper  vectors  of  G. 

Corollaries  (3.33)  and  (3.3^)  yield  bounds  for   M  )\[       These 

11^  oil 

bounds  are  attainable  when  i^^^^  =  af  [^^   =  a^f^^^  (a^O).   In  order  to  study 
the  effect  of  roundoff  on  these  bounds  the  following  vector  was  generated: 


fl     =   ^i(®i)"2 


5 


©.   =  2i  T  N+1  i  -  1,  .,.,  W 

vhere  s(©. )  is  the  machine  representation  of  |  sin  ^  as  generated  by  Illiac 
library  routine  T-5  which  has  an  accuracy  of  2"^^.   The  vector 
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f(i) 

Ji 


r^.^ 


V 


"l 


l?(l)l 


niji  11=   .0781  2500  00)  was  used  as  the  initial  yector  for  the  modified 
Chebyahev  method  and  successive  overrelaxatlon,  variant  one. 

Theorem  (3.7)  yields  an  upper  bound  for  Irijr  for  the  success- 
ive overrelaxatlon  method,  variant  two.   It  is  not  known  if  and  when  this 
hound  is  attainable.   However,  a  careful  examination  of  the  proof  of  theorem 
(3.7)  shows  that 


4 


vrhere 


rr  =  Ji  r(^2k^)  -  r'  ^^^-M'  -  (^^^.i^^)  -r'  ^"2^^^))'] 


1+A-|I^ 


-> 
when     ^       = 

-> 

q;    ^    ,   and 

''k 

Jl    f(S2k<P) 

'0 

5  [(s^Jti)  +(0  S2^_^(Ti))2  +  fsg^^,^(p)  .^2  s^j^n:))^^     (5.6) 


6 

ki 


when  £q  =  ae   and,  hence,  is  greater  when  i^  =  a  ?  . 

In  view  of  the  above,  the  second  initial  vector  for  the  successive  overrelax- 
atlon method,  variant  two,  was  chosen  as 

]l    =  s(e.)-2« 

e.   =  21  -^  N+l       1  =  1,...,  Kj^ 


1.10 


©.   =   (21-1)  -:-  K+1      1  =  1,...,  N^ 


with 


!?"![  =  .0276  2135  86. 

The  six  runs  on  the  Illlac  were  as  follows: 

Run  l.a  -  modified  Chebyshev  method;  f*^-""^  =  u^^^ 

Run  l.b  -  successive  overrelaxatlon,  variant  one-  f  ^"^^  -  ^^^-^^ 

'     ^  0 

Run  l.c  -  successive  overrelaxatlon,  variant  two;  r    =     -^ 
Run  2. a  -  modified  Chebyshev  method;  T   =  &^^^ 

Run  2.b  -  successive  overrelaxatlon,  variant  one:  r^^^   =  ^(^) 

)0  J 

— =>      _^ 

Run  2.C  -  successive  overrelaxatlon,  variant  two;  f  =  ^' . 

Each  run  consisted  of  6OO  Iterations.   Let 
""^,1  =  "  yk,l  +  ^Jk^i+W   +  Jk,l+N^+l^-2     i  =  1,...,N^ 


Jk,N+l  "  ° 


"? 


"2 


k,l+N^   "    J'k,l+N^  +   ^Jk,i-1  ■'Jlcl^  ^  2        1  =   1,...,  N. 

then  after  each  Iteration  |  |j  J  |  ,   |  | J^  -  r  ^  |  |  ,  and  |  \7 ^\  |  were  printed. 

In  addition,  for  the  modified  Chebyshev  method  after  each  iteration  b   and 
_  2k 

^2k+l  ^^^^   printed. 

The  initial  vector  was  chosen  so  that  p  =  p  =  ...  =  0.   The 
computation  was  carried  out  in  the  following  manner: 

K'i     '    ^■'^ak  ®[(Jk-i,i.N^-Jk-i,i.N^.i'-2-7,.i,il  Vk-i,.i 

i=  1.  2,...,  N^,  jVi^^^,  =  0 


Ill 


Jk,i+N^ 


with 


=  2c 


=  0 


2k-.l   ®  L!Jk,i-l^;k,i)"2-J^_^^    -  V^^l,i-,H 

ij  1 

1=1,2,...,!.^,   Jk,. 

Jo,i+N^  =  ^J 0,1-1  '^Jo,i^  r 


-   2 


i  =  1,  2,...,  M  , 


Jo,o  =  ° 


for  the  modified  Chehyshev  method  and  the  successive  overrelaxation  method, 

variant  one.  For  the  modified  Chebyshev  method,  c  =  b  with 

k    k 

.9695^  58295  <  \  <  1 
and  for  the  successive  overrelaxation  method,  variant  one  and  two, 

Cj^  =  b  =  .9695^  58295. 
The  pseudo-operation  "f   2"  indicates  a  right  shift,  and  "0"  indicates  a 
pseudo-multiplication  in  which  the  rounding  is  performed  to  take  advantage 
of  the  fact  that  the  next  operation  is  a  multiplication  by  two. 
Then  for  i  =  1, . . . ,  N^  and  k  >  0, 


U 


k,i  '  - 


^c 


2k 


-  2c 


Vk-l,i+N^  "^  Jk-l,i+N^+l^ 

^^2k  ®  f^Jk.i,i.N^-Jk-i,i.rT,.i)"2-yk.i,iJ 

^'^2k   f  ^Jk-l,i+N^  ■"  Jk-l,i+N^+l)  T  2  -  Jk.i^i] 


+  2c 


2kl 


Vk-l,i+N^  "''/k-l,i+N^+l^  .' 


Vk-l,i+N,  "^  Jk-l,i+W  +1^ 
-  o  —  — ± 1 


<  s-s-"*" 
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since  for  the  Illiac 


[2^     -  2xg)y]    <  2 


■ko 


'2 


X  T  2      <  2 


-ko 


The  same  analysis  holds  for  L(      I  Ci  -  i      w  ^   t   i^.^. 

^     jx   iux  |dj^^^^-[^  I  U  -  i>. . .,  W^j.  In  addition^  sin 

Pq  =  P._L  =  ••'  =  0,  ^^^.  =  y^^.  (i  =  1^  2,...,  N).   Then 


An  upper  hound  for  min  \\~j^\\     may  be  obtained  by  alloving  k- 
the  right  hand  side  of  (k.^k);   hence 


"^c  on 


"jn  \\jj,\\<^^+r 


-2 
1-p 


1(1- e^f 

2    

where  f  =  2b  -  1.   Then  substituting  the  bounds  for  | \f^^^ 

the  right  hand  side  of  (5-3),  we  have 

^^^^  M/i,!1  1 28,829  '  10"^^ 

In  addition,  from  theorem  (4.5),  we  have 


f 


-J 


a:id    I  I  #2), 


into 


(5.7) 


mm 
k 


iij.-;k-iii-<  ^r^^]l\\t''\\'^\\r^ 


(5.8) 


<  3567-10 


-12 


Instead  of  assuming  that  the  maximum  error  occurs  during  each  itera- 
tion, let  us  assume  that  J,  .   and  Y         fi  =  1      t\t-  v  v.  n"i  «„^  •  ^    ^  ^-^ 

k,i       ''k^i  ^    -L^  •  •  •  ^  ^\i^  k  >  Oj  are  independently 

distributed  random  variables  which  have  a  uniform  distribution  over  the  range 
(-3'2   ,3-2   ).  Then  if  the  average  value  of  a  random  variable  X  is  defined 
by  E  j  X  I  ,  we  haVe 
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r  3-2 


'3 
Then  substituting  the  upper  hound  into  (5.6)  and  (5.7),  we  h 


ave 


average  mln  |  ij^  -J^J  |  <  2059- 10"^  .  (5.I05 

5*3  The  numerical  results 


We  define 

>»   _  _ 


R'   -    10.    "^i" 


R^  =  -  log 


ifkl 


10 


ttr         -,_^        11 


H"'  =  -  log 
k       ^10 


I  If"  II 
1  I  j  0  II 

— 

where  J^  is  the  k   iterate  of  the  modified  Chebyshev  method;  fj^  is  the  k*^ 

iterate  of  the  successive  overrelaxation  method,  variant  one;  and  ?'"  is  the 
th  ^^ 

k   iterate  of  the  successive  overrelaxation  method,  variant  two.   In  figure 

I,  ve  haveR^,  r;;,  r-  fork  =  15,...,  ^50  when  J^^^ '  =  r^D"  =  ^(^)   and 
^3'  =  ■^.  The  curves  R',  r",  r"'  represent  the  maximum  theoretical  value  of 
^k'  ^k'  K    ^^   computed  from  corollaries  (3.33)  and  (3.3^)  and  theorem  (3.7). 
In  figure  II,  we  have  R^;.,  r;^,  rJ^'  for  ],  -  15,...,  ^50  when  f^^^'  =  J^^)"  =  jd) 
J"'  =J'.   For  figure  II,  the  curve  r'"  was  computed  from  (5.6).   Since  the 
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\    ^o    ?a 
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changes  in  R',  R^;  and  R-  appear  to  te  random  fluctuations  for  k  >  350,  ve  have 
not  plotted  R^  or  r;;  or  R-  for  Ic  >  ^50  nor  the  related  experimental  points 
in  figures  III  through  VII  for  k  >  i+50. 
We  now  define 

"i  =  -^^^10  llJi-Ji-ilU 

K    -  -^°gio  IIJk-K-iM, 
^k    =  -^°%o  lljk-jk'-ill- 

In  figure  III,  we  have  displayed  D^^,  D"  D"'  (k  =  15,  30,...,  V,0)  when 
^0         -  Jo    =  ^    ^^d  JJ'  =  ^.   The  curves  D',  D%  D'"  represent  the  maxi- 
mum theoretical  value  Of  D',  B'^,   D-  as  computed  from  (3.I7).  (3.18),  and  (3.20) 
respectively.   In  figure  IV,  we  have  D^;.,  D^,  D'"  whenj^^^'  =  f (l)"  =  l(l)  ^nd 
y^  =J'.  The  curves  D'  and  D"  were  computed  from  (3. 19).  A  computation  simi- 
lar to  the  one  used  in  computing  (5.6)  shows  f or  d'  =  a^ 

"^^"^^-^"   =  J^    ^''2k(^)  -  J2k-2(^))'  -   (jgk.K^)  -  J2k-l^^ll^oll 
where  (5.11) 


J  /-^  =  S  ,-     -2 


CH)  -  \([I)  -^  f'  2^.i(fi) 

a  m-1 

In  figure  IV,  D"'  was  computed  from  (5,11). 
In  figure  V,  we  have 

K  =  -i°8,o  ||?r|| 


■10  I  i*k 

^"  =   -  i°«io  I  l^-k 
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.  iQ    -JO    /a 
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(t  =  15,  30,...,  1^50)  for  runs  l.a,  l.b,  l.cj  and  in  figure  VI,  wq  have  I^, 
^k'  K'    ^^^  ^^^^   2-^'  2.l3,  2.C.  Since 


'2  ^jk     /k-1^   -  ^^2k+l 


K'?. 


1  -  2c 


2k+l 


2c 


^(2)  _^{2)    ,,2 


2k+l 


lirr'-jifiii^    (5.12) 


vhere 


r , 


For  the  numerical  example  presented  here,  D,  =  I  and  D  =1  so  that  (5.12) 
yields  a  bound  for  the  residual  vector.  Then  using  (5.12)  and  the  expansions 
^°^  J  k  ~  Jk-1  ^'^^     Jk   "  )\-l     ^^  section  (3.7)  we  are  able  to  determine 
uppef  bounds  for   |  [i^^^  |  |   for  the  three  methods.   In  figures  V  and  VI,  the 
curves  L',  L",  and  L"'  represent  the  theoretical  upper  bounds  for  L/,  L",  and 

Till 


Let  us   define 

B;      =      10^^ 
k 

Ji 

—      y 

B»     =      10^^ 

P 

-    Tk 

B^'    =      10^^ 

-    \fi 

Where  J^  is  the  k   iterate  of  the  modified  Chebyshev  method  if  the  calcula- 
tion were  performed  exactly  and  J^ J^ '  =  a^<l);  j^^   is  the  k"^^  iterate  of  the 
successive  overrelaxation  method,  variant  one,  if  the  calculation  were  performed 
exactly  and  /q^^  "  =  «/i^^''  ^^^    f^    i^  the  k^^  iterate  of  the  successive  over- 
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relaxation  method,  variant  two,  if  the  calculations  were  performed  exactly  and 
/o"=  ^f2'      ^"^   ^^g"^^  ^^>  ^^  have  plotted  B^,  B^J,  and  Bj^'  for  k  =  I5,  30,...,1|5Q 

5.^  Discussion  of  the  numerical  results 

From  figure  VII,  ve  are  able  to  observe  the  growth  and  behavior  of  the 
roundoff  errors.  We  note  that  for  all  three  methods,  the  contribution  of  the 
roundoff  error  is  approximately  the  same  for  k  <  315-  This  is  not  too  sur- 
prising since  the  only  difference  between  the  modified  Chebyshev  method  and 

the  successive  overrelaxation  method  is  in  the  computation  of  the  b  's  which 

k 

which  vary  from  b  by  at  most  k%.     We  would  expect  that  the  contribution  of  the 
roundoff  error  would  be  approximately  the  same  also  for  315  <  k  <  1^50.  How- 
ever, we  note  that  for  k  fixed  and  greater  than  315,  there  is  considerable 
variation  between  B^,  B,;;,  and  B^'  .   It  is  not  known  whether  this  is  due  to 
statistical  variation  or  some  other  cause. 

Equation  (5.6)  yields  an  upper  bound  for  min  | |?  | |  which  is  actually 
the  maximum  contribution  of  the  roundoff  error  and  is  given  by  (5.7)  as 
2883-10   .  From  figure  VII,  however,  we  see  that  for  k  between  I05  and  315, 
Bj^,  B«,  and  B^^'  vary  between  760- lo'^^  and  879-10- ^1.  Thus  the  contribution 
for  k  in  this  range  is  only  about  |  as  great  as  indicated  by  the  upper  bound 
for  min  |jjj^||.  If,  however,  we  consider  the  roundoff  errors  to  be  independ- 
ently distributed  random  variables  which  have  a  uniform  distribution  over  the 
range  (-3-2   ,3-2   ),  the  upper  bound  for  the  average  min  | |r  11  is  by 

(5.9)  1664-10    which  is  approximately  twice  as  great  as  B'  or  B"  or  B"'  for 

k     k     k 

k  between  105  and  315.  For  k  between  330  and  ^50,  we  have 


max         [B^,   b;^,  B^']  =  1652- 10*^^ 


k=330,...,450 
and  this  occurs  for  B^  when  k  =  36O.  Thus  for  k  in  this  range,  the  maximum 


12k 


contribution  of  roundoff  agrees  very  closely  with  the  upper  bound  for  the  aver- 

:^ 
age  min  \\J^\\.     We  note  that      max      [B'   B"  B"']  never  exceeds 
1664-10-".  k=15,...,U50   >^-     - 

In  figure  II,  we  see  that  for  k  =  I5,...,  325  the  experimental  points 

^k'  ^k'  ^k  ^Sree  very  closely  with  the  theoretical  curves  R',  R",  and  R'". 

However,  for  k  =  255,  we  note  that  R^  and  R^  depart  from  the  theoretical  values 

whereas  Rj^'  continues  to  agree  approximately  with  R'"  until  k  =  3i^5. 

We  shall  now  explain  why  R"'  agrees  more  closely  with  R'"  than  R' 

'^  "  k 

agrees  with  R'    or  RJJ  agrees  with  R".      It   is   not  difficult  to  see  that 


,r9 


Ro.)  -  B,  =  log,,  (1 .  ^^y^)  ■ 

Thus  the  difference  between  the  theoretical  curve  and  the  experimental  points 

is  a  function  of  the  relative  error;  and,  hence,  R(k)  -  R  may  be  large  even 

though   I  IJ^I  I  -  I  I  r  I  I    is  small.   From  figure  VII  we  see  that  for  k<  315 

I  Jj  I  ~  I  Ij'J  I    for  all  three  methods  and 


there  is  very  little  variation  in 


i^^^  Kjk  I  I  >  I  lyil  I   ^^^  I  i  Jk  I  I  >  I  Ijkl  h  R'"  (^)   agrees  most  closely  with 


As  a  direct  consequence  of  theorem(3.^)  for  k  fixed,  we  have 
max      R'(k)  >       i^ax     R"(k). 


kT'iM°  ii4'^iMo 


The  question  arises  whether  this  result  is  anulled  by  roundoff  errors.   As  was 

pointed  out  in  section  (5.2),  the  maximum  of  R'(k)  and  R"(k)  is  attained  when 

^0   ^  ^Jl        (^^^)-      From  figure  II  we  note  that  for  k  =  15,...,  315  R'  >  R". 

For  k  >  315  neither  R^^  nor  Rj^  is  uniformly  better  than  the  other  with  respect 

I  i  f  I  I 
to  k.  Nevertheless,  to  reduce  —^jjr     to  lO"^,  the  modified  Chebyshey  method 

I  I  Jo  II 

required  13^  fewer  iterations  than  the  successive  overrelaxation  method,  variant 
one. 
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In  fignjre  I  we  observe  that  for  k  =  15, ...,  3^5  r"  >r'.  jhe  rea- 
son  for  this  is  that  for  ^^^^  =  4'^^  max   R-  and     max    R,"  are  not 

obtained.   Thus  we  see  that  although  the  modified  Chebyshev  method  guarantees 
fewer  iterations  for  a  given  accuracy  the  successive  overrelaxation  method, 
variant  one,  may  actually  require  fewer  iterations  for  a  given  accuracy. 

In  both  figures  I  and  II  we  see  that  for  k  =  15,...,  330  R"  >  R'" 
and  R^  >  R^'  and  that  |  |  JJ^'j  |  actually  increased  during  the  early  iterations. 

Indeed,  for  run  l.c,  ^7  iterations  were  necessary  so  that  U^^- 1  I  <  1,  and  for 

I  I  r"<\  I    ■ ' 
jrun   2.C,  95  iterations  were  necessary  so  that  'i  JO'' 


IJS' 


<  1. 


Comparing  run  l.b  with  l.c  and  run  2.b  with  run  2.c,  we  see  that  by  beginning 
the  iteration  process  in  a  very  simple  fashion,  i.e.  y^^^   =  D""*"  (y^^^  -  S^^-*-^) 
the  number  of  iterations  necessary  so  that  the  initial  error  is  reduced  by  a 
fixed  percentage  is  considerably  diminished. 

In  figure  III  and  IV  we  observe  the  behavior  of  the  difference  vector 
J'k  ~  Tk-1'  ^°^  ®^^^  °^  *^®  three  methods.   In  figure  IV  we  note  that  there  is 
more  agreement  between  the  experimental  points  and  theoretical  curves  than  in 
figure  II.  A  possible  explanation  for  this  is  that  in  computing  the  difference 
vector  a  cancellation  of  roundoff  error  takes  place  which  does  not  occur  in  the 
computation  of  the  error  vector  n  .   This  is  verified  in  the  calculation  of 
the  upper  bounds  for  min  |  l]"j^  |  |  and  min  \\X:^~   T  _^  i  |  since  from  (5.7)  and 

(5.8)  we  see  that  the  upper  bound  for  min  |  |  ^^  ~  Jk-i  I  I  ^^  ^^0""t  k  the  upper 

bound  for  min  |  |  f ,  |  |,  and  the  upper  bounds  describe  the  maximum  contribution 
k    -^^ 
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of  the  roundoff  errors. 

For  some  numerical  problems  one  wishes  to  iterate  until  the  length 
of  the  error  vector  \\f^-t\\     is  as  small  as  possible.  If  we  base  our  ter- 
mination  procedure  on  |  \f^  - J^_^  |  |  ,  the  questions  arises  when  do  we  stop 
iterating.   In  both  figures  III  and  :I7  we  see  that  D^,  D^;,  and  D-  tend  to  in- 
crease and  then  "level  off",  i.e.  remain  relatively  constant.   If  we  compare 
figure  I  with  figure  III  and  figure  II  with  figure  IV  we  note  that  the  leveling 
off  occurs  for  |  \f^  -  f^^^  |  |  after  the  leveling  off  of  |  |  Jj  |  .   For  any 

terminating  procedure  which  is  based  on  the  "leveling  off"  of  the  I  I  f  -  r    II 

"y'k  ^k-l'l 
curve,  more  iterations  are  involved  than  would  be  warranted  by  the  decrease  in 

the  quantity  |  |  J^  -  x  i  |.  The  question  arises  whether  we  would  not  tend  to 
overiterate  if  we  based  our  termination  procedure  on  |  1  ?j^  |  |.  From  figures  V 
and  VI  we  see  that  the  behavior  of  |  |  ?  |  |  is  very  nearly  the  same  as  that  of 
"Jk  ~  Jk-1  I  I  ^"^^   ^®^^^  ^^e^s  is  no  advantage  to  be  gained  in  spending  the 
additional  time  to  calculate  |  |  ?J  |  .   It  is  not  surprising  that  I  I  ? 
I  ijk  ~J°k-l  I  I  ^^^®  similar  behavior  for  this  problem  in  view  of  (5.12). 

We  now  wish  to  consider  how  to  easily  sense  the  leveling  off  of 
'  '  Jk  ~  Jk-1   '  I'   ^^®  termination  procedure  which  has  been  used  for  s( 
ical  processes  is  to  continue  to  iterate  until  some  quantity  whic:i  ordinarily 
decreases  increases  or  remains  constant.  Although  it  appears  from  figure  III 
that  I  I J^  -  Jk.l  I  I  decreases  as  k  increases,  we  see  from  figure  IV  that  for 

k<  15   lljk"  Jk-lll  actually  increases.   Thus  the  aforementioned  termina- 
tion procedure  would  not  be  a  very  good  method  here. 

For  the  termination  procedure  we  now  wish  to  propose  we  are  unable 
to  give  a  rigorous  argument;  however,  for  this  example  it  seemed  to  work  suf- 
ficiently well.  From  (5. 10)  we  have  an  upper  bound  for  the  average 


■  1  II  and 
k  '  ' 


some  numer- 
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"k"  "Jk  /k-lH-  Sence  we  are  reasonably  assured  that  ||r  -f      n  „iii 
be  less  than  or  equal  to  the  bound  given  by  (5. 10).  Even  ii     \\f    Sf        |[ 
is  smaller  than  the  .Inlmum  given  by  (5.10)  one  should  continue  t^  Iterate  as 
long  as  ||;^-/,.J1   decreases.   If  ||J,-J^_J|  does  not  change  or  In- 
creases then  one  should  terminate  the  Iteration  process  since  it  is  not  known 
"''"°  lljk  -/k-lll  "1"  a«=rease  again.   In  the  table  below,  we  have  recorded 
the  value  of  k  for  which  the  iteration  process  would  have  been  terminated  and 
llJkl  I   and  I  IJ^  -J^_^   1 1  if  this  iteration  procedure  had  been  used  for  the 
six  runs. 


TABLE  II 

Run 

Terminating  value 

of  1: 

% 

l.a 

304 

810-10"^^ 

l.b 

291 

6ko  -10"  ■'■■'- 

l.c 

382 

702  •lo"-^-'- 

2.a 

309 

90l'l0"-'--'- 

2.b 

330 

1073  •10"-'-^ 

2.C 

366 

336-10"^^ 

lljk-lll 

157- 10"  ■'■■'- 

181. 10'^^ 

6.10-^1 


2  -10 


-11 


20-10 


■11 


24-10 


-11 


^  We  note  from  table  II  that  by  using  this  terminating  procedure 

IIJJI  ^for  each  of  the   six  runs   is    less  than  the  upper  hound  for  the  average 
"^^      lllkll      gi^e°  ^y   (5.9),   that   is,    average     min      |  |/j^|  |   <      1664-10-^1. 

5.5  Conclusions 

In  chapter  III  several  methods  are  derived  and  discussed  for  solving 
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Ai?=  i^^when  A  i^  syiometric,  positive  definite,  possesses  property  (a)  and 
ordering  ^^.  m  this  chapter  ve  have  shovn  the  results  of  using  these  methods 
for  solving  a  particular  example  in  which  a  high  speed  computer  was  used  for 
performing  the  calculation.   In  section  (5.I)  three  questions  vere  posed  with 
respect  to  the  numerical  experiment;  we  shall  now  discuss  these  questions  in 
the  light  of  the  numerical  results. 

The  first  question  asks  how  the  theoretical  results  of  chapter  III 
showing  the  relation  between  the  various  methods  is  affected  by  roundoff  errors. 
From  figure  II  we  note  that  for  k  <  315, 

^k  >  «k  >  K  (5.13) 

vhich  is  as  predicted  by  the  analytic  results.  However,  for  315  <  k  <  U50, 
(5.13)  no  longer  holds  true  and,  hence,  the  roundoff  errors  vitiate  the  analy- 
tic results  of  chapter  III. 

The  question  then  arises:  for  a  given  problem,  which  of  the  three 
methods  Should  be  used?  Let  us  assume  that  after  k  iterations  the  effects  of 
roundoff  error  is  the  same  for  all  three  methods.   From  a  stochastic  model  it 
mj   be  Shown  that  the  expected  value  of   |  | J^  -   Jj  |2  ^,   ^^^^^  ^^  ^^^   ^^_ 
pected  value  of   |  |JJ  f  -    |  |f  J  f,      m  section  (5.4)  it  was  observed  that 
for  k  <  315  the  effect  of  roundoff  as  measured  by  the  quantity  of 

ll/kll-  iijkll 
was  substantially  the  same  for  all  three  methods.  This  is  the  basis  for  the 
assumption  made  above.  Then  under  this  assumption  since 

and  since  for  k  and  the  length  of  the  initial  vector  fixed   |  |J  -  x | |   is 
smallest  for  the  modified  Chebyshev  method,  the  modified  Chebyshev  method 
should  be  used  in  preference  over  the  successive  overrelaxation  method,  variants 
one  and  two. 
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In  question  two  ve  query  how  realistic  are  the  hounds  given  in 
Chapter  IV  for  the  numerical  process.  If  ve  compare  the  non- stochastic  upper 
bound  given  hy  (5.7)  for  min  | |  f  J  |  with  the  experimental  values  of  ( [J J  [ 
given  in  table  II,  then  ve  see  the  bound  is  too  large  by  a  factor  between  2.7 
and  8.6  vhereas  the  stochastic  upper  bound  of  (5.9)  is  too  large  by  a  factor 
between  1.6  and  5.0.  Thus  the  stochastic  upper  bound  is  more  realistic  than 
the  non-Btochastic  upper  bound  for  min   |  |J J  | .  The  non-stochastic  upper 

bound  given  by  (5.8)  for  min  Mr    -   r    11^^   ^4.1- 

;f    \j      )   j-ox   min  \\ij^       -    J ^_^\\      exceeds  the  experimental 

values  of  |  J^^  -  /k^il  I  in  table  II  by  a  factor  betveen  2.0  and  I78  while 

the  stochastic  upper  bound  is  too  large  by  a  factor  between  1.1  and  IO3.  Thus 

the  stochastic  upper  bound  for  min  ||/^  -  J^_  J  |  agrees  more  closely  with 

I  ITk  ""Jk-ll  I  ^^^"^  ^^^   non- stochastic  upper  bound. 

The  last  question  asks  for  a  good  terminating  procedure  when  we  are 

interested  in  acquiring  as  much  accuracy  as  possible.  In  the  preceedlng 

section,  a  procedure  was  given  for  terminating  the  iteration  process  when  it 

was  desired  to  make  |  |  Jj^  -  f  |  |  as  small  as  possible  in  the  fewest  number  of 

iterations.  This  procedure  seemed  to  vork  adequately  for  the  example  given  in 

this  chapter.   It  would  be  desirable  in  future  investigations  to  establish 

rigorously,  if  possible,  a  terminating  procedure  with  optimal  properties. 
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Appendix 
A.l  Basic  matrix  relationships 

For  an  n  X  m  matrix,  the  elements  will  be  denoted  by  a  .  or  ^A  ?   .  An 
th   ,               -4  Ij    /  jij 

n   order  column  vector  f   has  elem-^^nts  f       f  ^    a 

j-  i^^  exem_nxs  ]^>  ^2.'  '"  fn'      ^   ^°^  vector  is  denoted 

"%>  * 
as  9     . 

Let  q;^)   =  J^  ^^^^,  ther,  1 1/|  |2  ,  (J,j?),  j,,  ..,pp,,  ^„^^„  |^|^  ^^^ 
"lower  bound"  \k\^   are  defined  as  follows: 

|A|   =   Max    iJALlL 
"   ilflMo   |||||    ' 

and 

Thus  If  A  Is  symmetric,  |a|^  Is  the  largest  proper  value  of  A  in  magni- 
tude, and  |A|^  is  the  smallest  proper  value  of  A  in  magnitude.  Some  basic 
relationships  are  given  in  [7].  They  are  as  follows: 

"A  Vs^  ■■•  Vkll  S  li/lll  *  ■••  -  l/jl  (A,l) 

l*"'lu  =  \^\t  (A.2) 

l^^lu   =   l«l  |A|u  (A.3.a) 

1°^!?   =   l"l  lAy  (A.3.b) 

^^'\        >   |AI^  -  |B|„  (A.5) 


K---  ^=lu    <   Klu  Usl^  ■■■  |A 


k'u 


(A. 6) 


Furthermore!  if  A  is  such  that 


/or  all  i,j  then 
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a.  .  <  C 


l^lu  S  ^'^     ■  (A.7) 


Also,  if  A  is  the  transpose  of  A,  then 


*, 


and 


I*  lu  =   lA|u  (A.8.a) 

l*\  '      M/  (A.8.b) 


|A*A|^  =  |AA»1^  =   |A|2  .   (,.5,^) 

|A*A|^  =  |AA*|^   =   lA|j  .  (A.9.b) 


If  0  is  orthogonal,  then 


l^lu  =  1^1/  =  1  •  (A.10) 

Since 

l|Ay|l   =   li/ll  -jMU    for   ||?|i  ^0, 


•>!  I   ^1  |->l 


II^^M    <    M^M    |A|,  (A.ll.a) 

and  thi9  obviously  holds  true  for   |  |^|  |    =  0.     Smilarly,   we  have 

I  1^1  I   >|A|^    M^ll    .  (A.ll.b) 

A.2     The  solution  of  a  finite  difference  equation 

The  following  lemma  is  well  known: 
I^emma  (A.l);      If 

^k+l  +  ^^k  +  ^^k-l  =  \  (k  >  1) 


then 
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\       ^2 


k-1    k-1 


""-  =  5^-^^  ^1-^1^2-xr^ 


=  k\ 


1   "2 
k-1 


1    '2 
k-1 


k-1  \^"J  -  x^"J 
y  +  Z   -1     2 


j=l    '^1 


1  yi  -  (k  -  i)xJyo+"Z  (k  -  j)xf-JS 

j=l 


^J    ^  *  \ 


if  x.^  =  X2 


where  \^   and  X^  are  the  roots  of  the  equat 


ion 


>».  +  a\  +  "b  =  0. 


A.  3  A  usefijil  inequality 
Lemma  (A. 2): 


Max 
O<0<T 


sin  kO 


sin  0 


<  k. 


Proof: 

For  k  =  2,     s1il20  ^  p  cos  Q  sin  Q  .  sin  29 

sin  9     "^^    sin  9 >        ^^^>   hence,    .   ^''^ 


Assiming 


sin(k-l)9 


sin  9 


sin  k9 
sin  9 


<  (k  -  l)  ,   then  ve  have  since 


sin  (k-l)9 
sin  9 


cos  9  +  cos(k-l)9, 


<  2 


sin  k9 
sin  9 


<  k 
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